Beyond the basics

If you are looking for information about purchasing and deploying sniffers you need: Setup and Installation
Location, access and basic capture handling is covered in Accessing and Getting Captures
Web GUI front end is in Browser Access

Skip to: PacketWave Analysis | High-Rate Direct-Access Capture Daemon

Dealing with large captures

The captures by default are for an hour each.
So you might want to chop them up.

You can do this with editcap which is included in the image:

user@workstation:~$ man editcap
EDITCAP(1)              The Wireshark Network Analyzer              EDITCAP(1)

NAME
       editcap - Edit and/or translate the format of capture files

SYNOPSIS
       editcap [ -A <start time> ] [ -B <stop time> ]
       [ -c <packets per file> ] [ -C <choplen> ] [ -E <error probability> ]
       [ -F <file format> ] [ -h ] [ -i <seconds per file> ] [ -r ]
       [ -s <snaplen> ] [ -S <strict time adjustment> ]
       [ -t <time adjustment> ] [ -T <encapsulation type> ] [ -v ] infile
       outfile [ packet#[-packet#] ... ]

       editcap  -d  |  -D <dup window>  |  -w <dup time window>  [ -v ] infile
       outfile
 . . .
       -i  <seconds per file>
           Splits the packet output to different files based on uniform time
           intervals using a maximum interval of <seconds per file> each. Each
           output file will be created with a suffix -nnnnn, starting with
           00000. If packets for the specified time interval are written to
           the output file, the next output file is opened. The default is to
           use a single output file.
 . . .
       -s  <snaplen>
           Sets the snapshot length to use when writing the data.  If the -s
           flag is used to specify a snapshot length, packets in the input
           file with more captured data than the specified snapshot length
           will have only the amount of data specified by the snapshot length
           written to the output file.
 . . .

A worked example of chopping up

In this I chopped up a 1 hour capture into 10 minute (600 second) chunks.
The original file is compressed (.pcap.gz) so I compressed the chopped files and you can see they amount to the same size:

user@workstation:/Windows/E/Projects$ du -csh eth1_00046_20141119142317.pcap.gz 
65M     eth1_00046_20141119142317.pcap.gz
65M     total
user@workstation:/Windows/E/Projects$ editcap -i 600 eth1_00046_20141119142317.pcap.gz eth1_00046_20141119142317-chop.pcap
user@workstation:/Windows/E/Projects$ du -csh eth1_00046_20141119142317-chop_0000*
37M     eth1_00046_20141119142317-chop_00000_20141119142316.pcap
56M     eth1_00046_20141119142317-chop_00001_20141119143316.pcap
43M     eth1_00046_20141119142317-chop_00002_20141119144316.pcap
55M     eth1_00046_20141119142317-chop_00003_20141119145316.pcap
49M     eth1_00046_20141119142317-chop_00004_20141119150316.pcap
40M     eth1_00046_20141119142317-chop_00005_20141119151316.pcap
16K     eth1_00046_20141119142317-chop_00006_20141119152316.pcap
278M    total
user@workstation:/Windows/E/Projects$ gzip -v eth1_00046_20141119142317-chop_0000*
eth1_00046_20141119142317-chop_00000_20141119142316.pcap:        76.7% -- replaced with eth1_00046_20141119142317-chop_00000_20141119142316.pcap.gz
eth1_00046_20141119142317-chop_00001_20141119143316.pcap:        76.9% -- replaced with eth1_00046_20141119142317-chop_00001_20141119143316.pcap.gz
eth1_00046_20141119142317-chop_00002_20141119144316.pcap:        76.8% -- replaced with eth1_00046_20141119142317-chop_00002_20141119144316.pcap.gz
eth1_00046_20141119142317-chop_00003_20141119145316.pcap:        76.8% -- replaced with eth1_00046_20141119142317-chop_00003_20141119145316.pcap.gz
eth1_00046_20141119142317-chop_00004_20141119150316.pcap:        76.7% -- replaced with eth1_00046_20141119142317-chop_00004_20141119150316.pcap.gz
eth1_00046_20141119142317-chop_00005_20141119151316.pcap:        76.6% -- replaced with eth1_00046_20141119142317-chop_00005_20141119151316.pcap.gz
eth1_00046_20141119142317-chop_00006_20141119152316.pcap:        73.1% -- replaced with eth1_00046_20141119142317-chop_00006_20141119152316.pcap.gz
user@workstation:/Windows/E/Projects$ du -csh eth1_00046_20141119142317-chop_0000*
8.5M    eth1_00046_20141119142317-chop_00000_20141119142316.pcap.gz
13M     eth1_00046_20141119142317-chop_00001_20141119143316.pcap.gz
10M     eth1_00046_20141119142317-chop_00002_20141119144316.pcap.gz
13M     eth1_00046_20141119142317-chop_00003_20141119145316.pcap.gz
12M     eth1_00046_20141119142317-chop_00004_20141119150316.pcap.gz
9.2M    eth1_00046_20141119142317-chop_00005_20141119151316.pcap.gz
8.0K    eth1_00046_20141119142317-chop_00006_20141119152316.pcap.gz
65M     total

Recombining to a single capture

The captures can be transferred off and inspected individually,
or you can re-combine them using mergecap or Wireshark (Merging capture files).

user@workstation:~$ man mergecap
MERGECAP(1)             The Wireshark Network Analyzer             MERGECAP(1)

NAME
       mergecap - Merges two or more capture files into one

SYNOPSIS
       mergecap [ -a ] [ -F <file format> ] [ -h ] [ -s <snaplen> ]
       [ -T <encapsulation type> ] [ -v ] -w <outfile>|- <infile> [<infile>
       ...]

DESCRIPTION
       Mergecap is a program that combines multiple saved capture files into a
       single output file specified by the -w argument.  Mergecap knows how to
       read pcap capture files, including those of tcpdump, Wireshark, and
       other tools that write captures in that format.

       By default, Mergecap writes the capture file in pcap format, and writes
       all of the packets from the input capture files to the output file.

PacketWave Analysis

In development

This tools is written in native C++ so it runs quickly.
The actual execution time on a 1 hour capture of ~300Mbytes size is around 4 seconds.
This increases around 1 minute for a ~5Gbyte file of the same period, so it scales roughly linearly with size.

Improvements:

This is being modified to separate the IP streams so giving separate analyses for each stream. ^Done
The tool currently reads the file into memory and then processes the time windows. I'm currently looking at modifying it to process as it reads, which should bring the time down.^Done
Also seeing if it can be included into the capturing daemon so it processes realtime. ^Progressing

Original LibreOffice Drawing

High-Rate Direct-Access Capture Daemon^NEW

A new capturing daemon using the PF_RING hardware accelerated capture technology is being developed.
This new daemon is entirely C++ and links directly in to the kernel drivers.

This means it will be far more accurate and has the ability to capture full packets to disc at full line speed without loss.

A capture daemon has already been tested and performs well even on the low resource Jetway Atom-based box.

Also this will include the analyser directly into the capture engine so stats will be produced real-time,
rather than post-processed from previously saved PCAP files

And this will include an AJAX interface so a remote browser shows the real-time analysis directly from the capture pipeline.
This will give the fastest and most intimate view of the real-time traffic in a really meaningful view.

Future topics / developments

Live capture access via remote desktop^Done
Graphical interface for statistics (web portal)^Done
Adding and updating the software (VPN repository access)^Done
Peer intelligent monitoring (central monitoring)^Queued
Multicast UDP latency/drop analysis^Queued

Images edited in gimp, flowcharts created in LibreOffice Draw.