Differences

This shows you the differences between two versions of the page.

--- pcaps:ixmgtool [2019/04/13 13:36]
veera [trisul_ixmgtool]
+++ pcaps:ixmgtool [2019/04/15 11:20] (current)
veera [Conclusion]
@@ Line 1: / Line 1: @@
-====== Merge multiple thin PCAP files into a single thick PCAP ======
+====== Merge multiple thin PCAP files into a single fat PCAP ======
 When you install Trisul Network Analytics , you get a free command line tool called ''trisul_ixmgtool''
@@ Line 7: / Line 7: @@
 ===== What is a FAT pcap file =====
-A FAT pcap file contains more unique flows and endpoints than a THIN pcap file.
+<note>A FAT pcap file contains more unique flows and endpoints than a THIN pcap file regardless of the actual bandwidth.
+</note>
 While testing NSM((Network Security Monitoring))  platforms we look for FAT pcap files because it stresses the memory and performance of algorithms.  Given a 10GB //thin// PCAP file with just 1 flow, and a 1GB //fat//  PCAP file with 100K flows - you should prefer the FAT file for testing.
@@ Line 17: / Line 18: @@
 ===== How is it different from mergecap  =====
-Mergecap  is a command line [[https://www.wireshark.org/docs/man-pages/mergecap.html|utility from the wireshark]] project.  It also combines multiple thin PCAP files into a single  fat PCAP file. But it preserves the timestamps, hence works to //fatten// the output PCAP if there is significant overlap in the time windows.
+Mergecap  is a command line [[https://www.wireshark.org/docs/man-pages/mergecap.html|utility from the wireshark]] project.  It also combines multiple thin PCAP files into a single  fat PCAP file. But it preserves the timestamps, hence works to //fatten// the output PCAP //only//  if there is significant overlap in the time windows.
-trisul_ixmgtool when run with the squish option , aligns the timestamps  of the files to the lowest timestamp and then processes the merge.  The following diagram illustrates the difference between mergecap and ixmgtool
+trisul_ixmgtool when run with the squish option , aligns the timestamps  of the files to the lowest timestamp and then processes the merge.  The following diagram illustrates the difference between mergecap and ixmgtool.
 {{:pcaps:ixmgtool.png |}}
+You can think of ixmgtool as combining the following three  operations
+  - Find the lowest timestamp from all the pcap files, and compute the deltas for each file
+  - Run ''editcap -t delta'' to transform the timestamps of each file
+  - Run ''mergecap'' on the transformed pcap files
-====== trisul_ixmgtool ======
+====== Using trisul_ixmgtool ======
 To get the free ixmgtool [[https://trisul.org/download|install Trisul Probe]] , you will find the trisul_ixmgtool in ''/usr/local/bin''
@@ Line 48: / Line 53: @@
 ===== Example run =====
-Say you have put 10 files in a directory  and you want to create a single FAT file.
+Say you have put 10 files in a directory  and you want to create a single FAT file.  If you are curious, we got these files from the good folks who run WRCCDC((The WRCCDC Cyber defense competition archives at https://archive.wrccdc.org/ ))
 <code>
@@ Line 97: / Line 102: @@
-To get a really fat pcap you can use the squish 10 option
+To get a **really FAT pcap** you can use the ''-squish10'' option. This creates 10 dummy flows for each flow by manipulating the source IP to 10 different IPs in the 10.0.0.x range.
+<code>
+unpl@unpl:~/wrccdc$ trisul_ixmgtool -squish10 -r . *.pcap -out really_fatone.pcap
+5000000 Packets  15005458762 Bytes Time Fri Mar  1 19:11:31 2019-475695
+EOF on wrccdc.regionals.2019-03-01.111203006460000.pcap, bye !
+EOF on wrccdc.regionals.2019-03-01.111159006450000.pcap, bye !
+EOF on wrccdc.regionals.2019-03-01.111147006420000.pcap, bye !
+8000000 Packets  23767761206 Bytes Time Fri Mar  1 19:11:32 2019-940185
+EOF on wrccdc.regionals.2019-03-01.111143006410000.pcap, bye !
+EOF on wrccdc.regionals.2019-03-01.111210006480000.pcap, bye !
+EOF on wrccdc.regionals.2019-03-01.111206006470000.pcap, bye !
+EOF on wrccdc.regionals.2019-03-01.111151006430000.pcap, bye !
+EOF on wrccdc.regionals.2019-03-01.111155006440000.pcap, bye !
+EOF on wrccdc.regionals.2019-03-01.111129006380000.pcap, bye !
+EOF on wrccdc.regionals.2019-03-01.111138006400000.pcap, bye !
+unpl@unpl:~/wrccdc$ ls -lh really_fatone.pcap
+-rw------- 1 unpl unpl 13G Apr 13 13:35 really_fatone.pcap
+</code>
+====== Conclusion ======
+trisul_ixmgtool  can be used to create FAT pcaps. These can be very useful for stressing NSM solutons.  Using the squish options you can create a mega thick PCAP file for testing by throwing all your PCAP testing files in single directory from varying timestamps and creating a single thick one.
+Hope this is useful to the NSM community.
+To get the tool (it is free).  Install the Trisul Probe package for your platform from  the [[https://trisul.org/download|Trisul Download page]]

Trisul Network Analytics Developer Zone

User Tools

Site Tools

Differences

Page Tools

Trisul Network Analytics
Developer Zone