User Tools
pcaps:ixmgtool
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
pcaps:ixmgtool [2019/04/13 13:41] veera [Example run] |
pcaps:ixmgtool [2019/04/15 11:20] (current) veera [Conclusion] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Merge multiple thin PCAP files into a single thick PCAP ====== | + | ====== Merge multiple thin PCAP files into a single fat PCAP ====== |
| When you install Trisul Network Analytics , you get a free command line tool called ''trisul_ixmgtool'' | When you install Trisul Network Analytics , you get a free command line tool called ''trisul_ixmgtool'' | ||
| Line 7: | Line 7: | ||
| ===== What is a FAT pcap file ===== | ===== What is a FAT pcap file ===== | ||
| - | A FAT pcap file contains more unique flows and endpoints than a THIN pcap file. | + | <note>A FAT pcap file contains more unique flows and endpoints than a THIN pcap file regardless of the actual bandwidth. |
| + | </note> | ||
| While testing NSM((Network Security Monitoring)) platforms we look for FAT pcap files because it stresses the memory and performance of algorithms. Given a 10GB //thin// PCAP file with just 1 flow, and a 1GB //fat// PCAP file with 100K flows - you should prefer the FAT file for testing. | While testing NSM((Network Security Monitoring)) platforms we look for FAT pcap files because it stresses the memory and performance of algorithms. Given a 10GB //thin// PCAP file with just 1 flow, and a 1GB //fat// PCAP file with 100K flows - you should prefer the FAT file for testing. | ||
| Line 17: | Line 18: | ||
| ===== How is it different from mergecap ===== | ===== How is it different from mergecap ===== | ||
| - | Mergecap is a command line [[https://www.wireshark.org/docs/man-pages/mergecap.html|utility from the wireshark]] project. It also combines multiple thin PCAP files into a single fat PCAP file. But it preserves the timestamps, hence works to //fatten// the output PCAP if there is significant overlap in the time windows. | + | Mergecap is a command line [[https://www.wireshark.org/docs/man-pages/mergecap.html|utility from the wireshark]] project. It also combines multiple thin PCAP files into a single fat PCAP file. But it preserves the timestamps, hence works to //fatten// the output PCAP //only// if there is significant overlap in the time windows. |
| - | trisul_ixmgtool when run with the squish option , aligns the timestamps of the files to the lowest timestamp and then processes the merge. The following diagram illustrates the difference between mergecap and ixmgtool | + | trisul_ixmgtool when run with the squish option , aligns the timestamps of the files to the lowest timestamp and then processes the merge. The following diagram illustrates the difference between mergecap and ixmgtool. |
| {{:pcaps:ixmgtool.png |}} | {{:pcaps:ixmgtool.png |}} | ||
| + | You can think of ixmgtool as combining the following three operations | ||
| + | - Find the lowest timestamp from all the pcap files, and compute the deltas for each file | ||
| + | - Run ''editcap -t delta'' to transform the timestamps of each file | ||
| + | - Run ''mergecap'' on the transformed pcap files | ||
| - | ====== trisul_ixmgtool ====== | + | ====== Using trisul_ixmgtool ====== |
| To get the free ixmgtool [[https://trisul.org/download|install Trisul Probe]] , you will find the trisul_ixmgtool in ''/usr/local/bin'' | To get the free ixmgtool [[https://trisul.org/download|install Trisul Probe]] , you will find the trisul_ixmgtool in ''/usr/local/bin'' | ||
| Line 48: | Line 53: | ||
| ===== Example run ===== | ===== Example run ===== | ||
| - | Say you have put 10 files in a directory and you want to create a single FAT file. | + | Say you have put 10 files in a directory and you want to create a single FAT file. If you are curious, we got these files from the good folks who run WRCCDC((The WRCCDC Cyber defense competition archives at https://archive.wrccdc.org/ )) |
| <code> | <code> | ||
| Line 97: | Line 102: | ||
| - | To get a **really fat pcap** you can use the squish 10 option | + | To get a **really FAT pcap** you can use the ''-squish10'' option. This creates 10 dummy flows for each flow by manipulating the source IP to 10 different IPs in the 10.0.0.x range. |
| Line 125: | Line 130: | ||
| ====== Conclusion ====== | ====== Conclusion ====== | ||
| + | trisul_ixmgtool can be used to create FAT pcaps. These can be very useful for stressing NSM solutons. Using the squish options you can create a mega thick PCAP file for testing by throwing all your PCAP testing files in single directory from varying timestamps and creating a single thick one. | ||
| - | The trisul_ixmgtool part of the Trisul NSM suite can be used for free to create FAT pcaps which can be very useful for stressing NSM solutons. We use this in Trisul NSM to help users download PCAPs of various investigations. | + | Hope this is useful to the NSM community. |
| - | Using the squish options you can create a mega thick PCAP file for testing by throwing all your PCAP testing files in single directory from varying timestamps and creating a single thick one. | ||
| - | Hope this is useful for the NSM community. | + | To get the tool (it is free). Install the Trisul Probe package for your platform from the [[https://trisul.org/download|Trisul Download page]] |
| - | + | ||
| - | + | ||
| - | To get the tool : Install the Trisul Probe package for your platform from the [[https://trisul.org/download|Trisul Download page]] | + | |
pcaps/ixmgtool.1555162910.txt.gz · Last modified: 2019/04/13 13:41 by veera
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
