10g packet capture 1U front

Overview

System Architecture

FMADIO10

The FMADIO10 packet capture device is our entry level full sustained line rate 10Gbit capture to cache packet capture / packet sniffer devices. It is a compact 1U 650mm deep chassis featuring 7.4 nanosecond resolution hardware packet time stamps and sub 100ns world time accuracy via PTPv2 or GPS. In addition there is 1TB of high bandwidth SSD flash storage which is written back into 16TB of raw magnetic disk drives. The system is unique by combining a hybrid SSD / HDD storage architecture to gain maximum cost savings with maximum disk storage and still be capable of sustained 1TB worth of line rate capture without any packet drops.

10g packet capture architecture

Features:


  • Sustained 10Gbit line rate capture to 1TB Cache
  • Compact 1U form factor
  • Hardware Packet Time Stamping
  • Sub 100ns accurate World time synchronization
  • PTPv2 Time Synchronization
  • PTPv2+PPS Time Synchronization
  • GPS+PPS Time Synchronization
  • 16TB of Raw hot swap Disk storage
  • 1TB of high speed SSD cache
  • x1 10G SFP+ Capture Port
  • x1 10G SFP+ Management Port
  • x1 1G RJ45 Management Port


FMADIO20

For full 10Gbps duplex line rate capture FMADIO20 packet capture device provides sustained line rate 20Gbit capture using 2x10G SFP+ 10Gbe ports. Housed in a compact 1U 650mm deep chassis featuring 7.4 nanosecond resolution hardware packet time stamps and sub 100ns world time accuracy via PTPv2 or GPS. This 20Gbe packet capture device has 2TB of high bandwidth SSD flash storage with over 20Gbps+ of sustained IO throughput. This 2TB of cache is written back into 16TB of raw magnetic disk storage. This system is ideal for medium utilization full duplex 20Gbps packet capture.

20g packet capture 1U architecture

Features:


  • Sustained 20Gbit line rate capture to 2TB of Cache
  • Compact 1U form factor
  • Hardware Packet Time Stamping
  • Sub 100ns accurate World time synchronization
  • PTPv2 Time Synchronization
  • PTPv2+PPS Time Synchronization
  • GPS+PPS Time Synchronization
  • 16TB of Raw hot swap Disk storage
  • 1TB of high speed SSD cache
  • x2 10G SFP+ Capture Port
  • x1 10G SFP+ Management Port
  • x1 1G RJ45 Management Port

hardware

Whats in the box

FMADIO10/FMADIO20

System includes the following items :


  • x1 1U FMADIO10 or FMADIO20 Packet sniffer system (A)
  • x1 GPS PPS -> PPS In SMA Coax Cable (B)
  • x1 110V Japanese/American Power connector (C)
  • x1 Rack mount Rail Kit (optional) (D)


(A) 1U FMADIO10 or FMADIO20 packet sniffer device

(outside chassis is identical for both models)

10g packet capture 1U front

(B) GPS PPS -> PPS In SMA Coax Cable

GPS PPS Out to PPS In SMA Cable

(C) 110V Japanese/American Power connector

Power Cable

(D) Rack mount Rail Kit (Optional)

Rack mount Rail Kit

Hardware Layout

Front Ports

The hardware interfaces at the front of the chassis are as follows:

10g packet capture rear front interfaces
How swap 3.5" Drive

Standard Hot swap 3.5" SATA drives of the internal RAID5 array. These are populated with Toshiba 4TB SATA drives resulting in a total of 16TB of raw magnetic disk storage.



10g packet capture rear front hotswap drives Close up of hotswap drive bay


Power Switch

Power button to turn on the device. To force a power off hold the button for 10 seconds.


Reset Switch

Hard system reset button, effective immediately.


USB 3.0 Port

Single USB 3.0 port.


Rear Ports

The hardware interfaces at the rear of the chassis are as follows:
(Note that FMADIO10 and FMADIO20 have slightly different ports)



FMADIO10
10g packet capture rear port interfaces
FMADIO20
20g packet capture rear port interfaces

Each port is described as below:


IPMI

IPMI port is a 10/100/1000M RJ45 ethernet network port used for out of band management. It provides system monitoring, serial port access and enables remote machine reboot via SSH or Web interface


1G Management

Primary RJ45 management / user interface for the device at 10M/100M/1Gbit speeds. Access is provided via HTTP/HTTPS and SSH.


10G SFP+/SFP Mgmt

High speed SFP/SFP+management / user interface running at 10Gbps. This is dual mode SFP/SFP+ port enabling both 1G SFP modules and 10G SFP+. Access is provided via HTTP/HTTP and SSH.


10G SFP+/SFP Capture

These are the 10Gbit capture interface. It is also dual mode SFP/SFP+ interface enabling 1G and 10G capture based on the clients requirements. FMADIO10 has 1 capture interface, while FMADIO20 has 2 capture interfaces.


PPS Out

Extreme accuracy (10ns <) 1PPS (one pulse per second) signal. It runs off a highly accurate TXCO (Temperature controlled crystal) enabling external devices to synchronize to FMADIO10/20 highly accurate world time. This is a 5V CMOS PPS signal, the rising edge indicating start of a second and is active for 8ms. Connector type is SMA female connector.


PPS In

Expects a 1PPS (one pulse per second) with the signals rising edge indicating the start of the second. Electrical characteristics are 5V into 50ohm to ground. This enables highly accurate packet time stamps, with extreme time synchronization accuracy with an external device such as Time Grandmaster. Synchronization accuracy is typically typically 10-20ns. This is a SMA female connector.


GPS PPS Out

GPS (Global Position System) 1PPS (one pulse per second) provides highly accurate world time generated from a simultaneous 22 satellites in geo-synchronous orbit. This highly accurate world time can be exported to other devices or connected back into the FMADIO10/20 capture system by connecting the "PPS In" to the "GPS PPS Out" port with the included SMA connector cable. This is a SMA female connector.


GPS Antenna

GPS (Global Position System) external active antenna SMA female connector. For maximum time accuracy, the antenna requires a 180 degree view of the sky.

Configuration

Network Configuration

Network port configuration can be achieved using a) the web interface, b) SSH command line interface(CLI). Using the Web interface is the easiest route, however in highly constrained network environments a pure CLI based configuration can be easier


Web Interface: Network Config


From the dashboard page, Start by selecting the configuration menu option from as shown below (highlighted in green).


10g packet capture configruation start
 

Then edit the network configuration`s IP/Netmask/Gateway/DNS setting as shown in the image below. After each field has been edited the system automatically saves and updates the system setting (save button is not required). After completing the update, refresh the web page to confirm the new settings.


10g packet capture configruation network web
 

Select the tools menu from the top toolbar, as shown in the image below.


10g packet capture configruation network web
 

And finally select the Power Cycle / Reboot button to restart the system


10g packet capture configruation network web
 
 

CLI Interface: Network Config


Modifying the network configuration setting in a restricted Colocation environment can be far easier to achieve via the command line. The first step is SSH into the system, change to the specified directory and view the current network settings, as shown below [email protected]:/tmp$ ssh [email protected] [email protected]'s password: _____ .___.__ 10G _/ ____\_____ _____ __| _/|__| ____ \ __\/ \ \__ \ / __ | | | / _ \ | | | Y Y \ / __ \_/ /_/ | | |( <_> ) |__| |__|_| /(____ /\____ | |__| \____/ \/ \/ \/ ============================================ -+ no user serviceable parts inside +- [email protected]:~$ cd /mnt/store0/etc [email protected]:/mnt/store0/etc$ cat network.lua -- auto generated on Tue Apr 14 10:38:13 2015 local Config = { ["sf0"] = { ["Mode"] = "disabled", ["Address"] = "192.168.1.2", ["Netmask"] = "255.255.255.0", ["Gateway"] = "192.168.1.1", ["DNS"] = "192.168.1.1", }, ["sf1"] = { ["Mode"] = "static", ["Address"] = "192.168.12.10", ["Netmask"] = "255.255.255.0", ["Gateway"] = "192.168.12.1", ["DNS"] = "192.168.12.1", }, ["eth0"] = { ["Mode"] = "static", ["Address"] = "192.168.11.75", ["Netmask"] = "255.255.255.0", ["Gateway"] = "192.168.11.1", ["DNS"] = "192.168.11.1", }, ["bmc"] = { ["Mode"] = "static", ["Address"] = "192.168.11.73", ["Netmask"] = "255.255.255.255", ["Gateway"] = "192.168.11.1", ["DNS"] = "192.168.11.1", }, } return Config In the example configuration file above, the network ports are mapped as follows

sf0 -> 10G SFP/SFP+ Capture Interface
sf1 -> 10G SFP/SFP+ Management interface
eth0 -> 1G RJ45 Management interface
bmc -> 1G RJ45 IPMI Interface

In the above example we see, sf1 -> 192.168.12.10/24, eth0 -> 192.168.11.75/24, and IPMI -> 192.168.11.73

This is confirmed by using the ifconfig command [email protected]:/mnt/store0/etc$ ifconfig eth0 Link encap:Ethernet HWaddr 00:25:90:FC:88:3C inet addr:192.168.11.75 Bcast:192.168.11.255 Mask:255.255.255.0 inet6 addr: fe80::225:90ff:fefc:883c/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:3779 errors:0 dropped:14 overruns:0 frame:0 TX packets:4466 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:756557 (738.8 KiB) TX bytes:4443144 (4.2 MiB) Memory:fbb00000-fbb7ffff lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:32962 errors:0 dropped:0 overruns:0 frame:0 TX packets:32962 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:10438860 (9.9 MiB) TX bytes:10438860 (9.9 MiB) sf0 Link encap:Ethernet HWaddr 00:0F:53:26:B9:10 inet6 addr: fe80::20f:53ff:fe26:b910/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:9216 Metric:1 RX packets:200000000 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:13600000000 (12.6 GiB) TX bytes:680 (680.0 B) Interrupt:26 sf1 Link encap:Ethernet HWaddr 00:0F:53:26:B9:11 inet addr:192.168.12.10 Bcast:192.168.12.255 Mask:255.255.255.0 inet6 addr: fe80::20f:53ff:fe26:b911/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:9216 Metric:1 RX packets:3976753 errors:0 dropped:0 overruns:0 frame:0 TX packets:117267670 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:280799142 (267.7 MiB) TX bytes:176538287574 (164.4 GiB) Interrupt:44 Steps to manually change the network configuration:


  1. 1) Modify/edit the /mnt/store0/etc/network.lua configuration file to the appropriate setting
  2. 2) Run $ sudo network_config.lua --nocal --updatebmc This updates the internal scripts and IPMI configuration flash.

  3. 3) Reboot the system $ sudo reboot

Upon reboot the system will now be configured with the updated network information. Note, its best to change network settings by logging into the system by the serial port, on the IPMI interface. The following example shows how to login via the serial console. $ ssh [email protected] [email protected]'s password: ATEN SMASH-CLP System Management Shell, version 1.05 Copyright (c) 2008-2009 by ATEN International CO., Ltd. All Rights Reserved -> cd system1/sol1 /system1/sol1 -> start /system1/sol1 press , , and then to terminate session (press the keys in sequence, one after the other) fmadio10G fmadio10-049 login: fmadio Password:****** _____ .___.__ 10G _/ ____\_____ _____ __| _/|__| ____ \ __\/ \ \__ \ / __ | | | / _ \ | | | Y Y \ / __ \_/ /_/ | | |( <_> ) |__| |__|_| /(____ /\____ | |__| \____/ \/ \/ \/ ============================================ -+ no user serviceable parts inside +- [email protected]:~$ cd /mnt/store0/etc [email protected]:/mnt/store0/etc$ cat network.lua -- auto generated on Tue Apr 14 10:38:13 2015 local Config = { ["sf0"] = { ["Mode"] = "disabled", ["Address"] = "192.168.1.2", ["Netmask"] = "255.255.255.0", ["Gateway"] = "192.168.1.1", ["DNS"] = "192.168.1.1", }, ["sf1"] = { ["Mode"] = "static", ["Address"] = "192.168.12.10", ["Netmask"] = "255.255.255.0", ["Gateway"] = "192.168.12.1", ["DNS"] = "192.168.12.1", }, ["eth0"] = { ["Mode"] = "static", ["Address"] = "192.168.11.75", ["Netmask"] = "255.255.255.0", ["Gateway"] = "192.168.11.1", ["DNS"] = "192.168.11.1", }, ["bmc"] = { ["Mode"] = "static", ["Address"] = "192.168.11.73", ["Netmask"] = "255.255.255.255", ["Gateway"] = "192.168.11.1", ["DNS"] = "192.168.11.1", }, } return Config [email protected]:/mnt/store0/etc$


Firmware Update

Firmware updates are simple and easy that enables the latest updates and system features. Please follow these steps to update the system.


Step 1) Download the latest firmware

Download the latest firmware image from the support web site.


Step 2) Select Firmware web page

Proceed to the "Tools" page on the capture device`s web page, as highlighted in green in the image below.


10g packet capture firmware update top
 
Step 3) Select the firmware binary to upload

Scroll down on the Tools page to find and select the "Choose File" firmware button (highlighted in green below)


10g packet capture firmware update select
 
Step 4) Upload the binary

After selecting the appropriate file (in this example fmadio10_20150623_1257.bin) click the upload button to transfer to the capture device. NOTE: this will not change the firmware on the device, only upload it.

The upload status is highlighted in blue below.


10g packet capture firmware update upload
 
Step 5) Verify firmware was uploaded

After successfully uploading the firmware, the web page will refresh and show a new entry in the firmware list, as shown in green below.


10g packet capture firmware update upload
 
Step 6) Select firmware and reboot

Select the new firmware image by clicking on the green icon (highlighted in green below). The green select button (it will remain highlighted).

After selecting which firmware to use, request the system to update then reboot (highlighted in blue below)


10g packet capture firmware update upload
 
Step 7) Confirm new firmware

It will take 1-2 minutes for the system to reboot and become active again. Once the system is online go to the tools page in the browser and confirm the currently active firmware image is the one just uploaded and selected.


System updates are easy and simple. The device is designed for regular updates as we are constantly adding additional features based on customer requests. This is NOT a standard redhat/ubuntu based linux system, it is a highly customized and controlled embedded linux device.

Think of our devices as Network Switch`s that do Packet Capture.

Change Hostname

Having the appropriate hostname can make server management alot simpler. By default all systems have a "fmadio-XXX" host name that is unique to each system deployed. Changing the host name is simple with the following steps


Step 1) Log into the system

SSH into the fmadio10 device $ ssh [email protected] [email protected]'s password: _____ .___.__ 10G _/ ____\_____ _____ __| _/|__| ____ \ __\/ \ \__ \ / __ | | | / _ \ | | | Y Y \ / __ \_/ /_/ | | |( <_> ) |__| |__|_| /(____ /\____ | |__| \____/ \/ \/ \/ ============================================ -+ no user serviceable parts inside +- [email protected]:~$


Step 2) Change to the configuration directory

[email protected]:~$ cd /mnt/store0/etc [email protected]:/mnt/store0/etc$


Step 3) Display the current host name

[email protected]:/mnt/store0/etc$ cat hostname fmadio10-049 [email protected]:/mnt/store0/etc$


Step 4) Set new hostname

Use your preferred editor or even command to edit the hostname file. For example:
[email protected]:/mnt/store0/etc$ sudo echo "my.new.hostname" > hostname [email protected]:/mnt/store0/etc$


Step 5) Power cycle the system

For the change to take effect, please reboot the system.
[email protected]:/mnt/store0/etc$ sudo reboot [email protected]:/mnt/store0/etc$ Connection to 192.168.11.75 closed by remote host.


Step 6) Complete

Next time you login the hostname is the new updated value, in this case "my.new.hostname" fmadio@my.new.hostname:/mnt/store0/etc$

RAID5 configuration

FMADIO10/20 can be configured as 12TB in a RAID5 configuration. The procedure is simple, but it destroys all data on the system thus care is required. In addition to the initial procedure, RAID5 systems require 8 hours to complete initialization of the drive. During this time the system performance is reduced.


Step 1) Log into the system

SSH into the fmadio10 device $ ssh [email protected] [email protected]'s password: _____ .___.__ 10G _/ ____\_____ _____ __| _/|__| ____ \ __\/ \ \__ \ / __ | | | / _ \ | | | Y Y \ / __ \_/ /_/ | | |( <_> ) |__| |__|_| /(____ /\____ | |__| \____/ \/ \/ \/ ============================================ -+ no user serviceable parts inside +- [email protected]:~$


Step 2) Issue disk format command

From the command line, enter the following options.
[email protected]:~$ cd /opt/fmadio/bin [email protected]:/opt/fmadio/bin$ ./format_raid.lua --raid5 fmad fmadlua Aug 11 2015 calibrating... 0 : 00000000d09daaff 3.5000 cycles/nsec Cycles/Sec 3499993855.0000 Std: 0cycle std( 0.00000000) loading filename [./format_raid.lua] done: 0 0 done 3.023745Sec 0.050396Min [email protected]:/opt/fmadio/bin$ Connection to 192.168.11.75 closed by remote host. Connection to 192.168.11.75 closed. [email protected]:~$ The system will now perform a series of system initialization and reboots. It will take about 5 minutes to complete.


Step 3) After about 5 minutes, login and check the RAID5 status

After about 5 minutes, the system will have completed the change. To check its status run the following SSH commands. [email protected]:~$ ssh [email protected] [email protected]'s password: _____ .___.__ 10G _/ ____\_____ _____ __| _/|__| ____ \ __\/ \ \__ \ / __ | | | / _ \ | | | Y Y \ / __ \_/ /_/ | | |( <_> ) |__| |__|_| /(____ /\____ | |__| \____/ \/ \/ \/ ============================================ -+ no user serviceable parts inside +- [email protected]:~$ sudo mdadm --detail /dev/md0 /dev/md0: Version : 1.2 Creation Time : Wed Aug 12 08:52:10 2015 Raid Level : raid5 Array Size : 11720662464 (11177.70 GiB 12001.96 GB) Used Dev Size : 3906887488 (3725.90 GiB 4000.65 GB) Raid Devices : 4 Total Devices : 4 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Wed Aug 12 08:58:46 2015 State : active, degraded, recovering Active Devices : 3 Working Devices : 4 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 64K Rebuild Status : 0% complete Name : fmadio10-049:0 (local to host fmadio10-049) UUID : a6d5fc4b:dbb7f274:7918601a:938c8451 Events : 202 Number Major Minor RaidDevice State 0 8 112 0 active sync /dev/sdh 1 8 64 1 active sync /dev/sde 2 8 96 2 active sync /dev/sdg 4 8 80 3 spare rebuilding /dev/sdf [email protected]:~$ The above shows the array is rebuilding. To get a better idea on its progress and completion time, check the following file. [email protected]:~$ cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] [raid0] md1 : active raid0 sdb2[0] sdi2[3] sdd2[2] sdc2[1] 468594688 blocks super 1.2 64k chunks md0 : active raid5 sdh[0] sdf[4] sdg[2] sde[1] 11720662464 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/3] [UUU_] [>....................] recovery = 0.9% (37383872/3906887488) finish=481.4min speed=133952K/sec bitmap: 1/30 pages [4KB], 65536KB chunk unused devices: [email protected]:~$ In the above example, there are 481minutes or about 8 Hours remaining to complete initialization


Step 4) Confirm web GUI is rebuilding

Point the browser to the dashboard page, to confirm the RAID5 array is rebuilding

10g packet capture raid5 rebuilding

The above image shows the RAID Status as degraded and rebuilding. It will take about 8 hours for this to complete.


Step 5) Confirm web GUI file system

Click on the "Files" browser tab to list all captures. There should be no captures, as the file system has just been formatted.

10g packet capture raid5 files
 

Step 6) Wait for RAID5 rebuild to complete

After about 8 hours, the RAID5 filesystem will be completely rebuilt. To confirm rebuild has completed successfully the below "clean" RAID Status (highlighted in green) shows the RAID5 filesystem has been successfully rebuilt, and ready for operation.

10g packet capture raid5 rebuild complete  


Step 7) Complete

System is now ready for all capture workloads.

RAID0 configuration

FMADIO10/20 can be configured as 16TB in a RAID0 configuration. RAID0 primary benefit is larger disk space and high performance. However a single disk failure will result in data loss. The following steps show how to configure the system for RAID0, it does not require an array rebuild and can be used immediately.


Step 1) Log into the system

SSH into the fmadio10 device $ ssh [email protected] [email protected]'s password: _____ .___.__ 10G _/ ____\_____ _____ __| _/|__| ____ \ __\/ \ \__ \ / __ | | | / _ \ | | | Y Y \ / __ \_/ /_/ | | |( <_> ) |__| |__|_| /(____ /\____ | |__| \____/ \/ \/ \/ ============================================ -+ no user serviceable parts inside +- [email protected]:~$


Step 2) Issue RAID0 disk format command

From the command line, enter the following options.
[email protected]:~$ cd /opt/fmadio/bin/ [email protected]:/opt/fmadio/bin$ ./format_raid.lua --raid0 fmad fmadlua Aug 11 2015 calibrating... 0 : 00000000d09da89a 3.5000 cycles/nsec Cycles/Sec 3499993242.0000 Std: 0cycle std( 0.00000000) loading filename [./format_raid.lua] done: 0 0 done 3.116626Sec 0.051944Min [email protected]:/opt/fmadio/bin$ Connection to 192.168.11.75 closed by remote host. Connection to 192.168.11.75 closed. [email protected]:~$ The system will now perform a series of system initialization and reboots, this will take about 5 minutes to complete.


Step 3) After about 5 minutes, login and check RAID0 initialization has completed

After about 5 minutes, the system will have completed the change. To check its status run the following SSH commands. [email protected]:~$ ssh [email protected] [email protected]'s password: X11 forwarding request failed on channel 0 _____ .___.__ 10G _/ ____\_____ _____ __| _/|__| ____ \ __\/ \ \__ \ / __ | | | / _ \ | | | Y Y \ / __ \_/ /_/ | | |( <_> ) |__| |__|_| /(____ /\____ | |__| \____/ \/ \/ \/ ============================================ -+ no user serviceable parts inside +- [email protected]:~$ sudo mdadm --detail /dev/md0 /dev/md0: Version : 1.2 Creation Time : Wed Aug 12 09:39:50 2015 Raid Level : raid0 Array Size : 15627549952 (14903.59 GiB 16002.61 GB) Raid Devices : 4 Total Devices : 4 Persistence : Superblock is persistent Update Time : Wed Aug 12 09:39:50 2015 State : clean Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Chunk Size : 64K Name : fmadio10-049:0 (local to host fmadio10-049) UUID : 42d3cf0d:92b4e95c:1cee3e93:fdd2558b Events : 0 Number Major Minor RaidDevice State 0 8 112 0 active sync /dev/sdh 1 8 64 1 active sync /dev/sde 2 8 96 2 active sync /dev/sdg 3 8 80 3 active sync /dev/sdf [email protected]:~$


Step 4) Confirm web GUI RAID status is clean

Point the browser to the dashboard page, to confirm the RAID0 array is clean (highlighted in green)

10g packet capture raid5 rebuild complete  


Step 5) Confirm web GUI file system

Click on the "Files" browser tab to list all captures. There should be no captures, as the file system has just been formatted.

10g packet capture raid5 files  

Step 6) Complete

System is now ready for all capture workloads.

iSCSI Storage

Standard FMAD10/20 systems can use up to 300TB of remote iSCSI target storage for long term packet capture. By default All systems are configured to utilize the local 16TB of 3.5" HDD storage in RAID0 or RAID5. This section describes how to configure a remote iSCSI backend storage device.


Step 1) Navigate to the Config page

Click on the "CONFIG" menu bar at the top of the screen as shown in green below.


Step 2) Enable iSCSI Backend

Select "Enable" from the drop down menu as highlighted in green


10g packet capture firmware update top
 
Step 3) Set iSCSI Target`s IP Address

Enter the full IP address of the iSCSI target device


10g packet capture firmware update top
 
Step 4) Set iSCSI Target Name

Enter the full iSCSI Target name, as highlighted in green below


10g packet capture firmware update top
 
Step 5) Format iSCSI Target Storage

Now need to re-format the storage system to utilize the iSCSI target. This is an un-recoverable operation that destroys all previous capture data on the system. THIS IS AN UNRECOVERABLE OPERATION. Start by selecting "iSCSI" from the Storage Mode menu as highlighted in green below.


10g packet capture firmware update top
 

Then click on the FORMAT button next to it. This will prompt with a confirmation dialog box and then a system message. This operation reboots the systems multiple times during storage initialization. It can take 5-30min depending on the size of the iSCSI Target device.


SNMP Setup with NAGIOS

Nagios XI is an excellent and popular system monitoring software used extensively thought the IT industry. Its extremely flexible and all FMAD devices now have full support. To setup and install FMAD device into NAGIOS follow the following steps.

1) Install the basic Linux SNMP target as shown in red below. Use all the default settings.

10g packet capture manual nagios setup 0
 
2) Enter FMAD management port IP address (as shown in Red)

This sets up basic Linux SNMP monitoring.
10g packet capture manual nagios setup 0
 
3) Continue and finish using the default settings

10g packet capture manual nagios setup 0
  10g packet capture manual nagios setup 0
 
4) Generate custom Nagios FMAD configuration file

Generate a Nustom nagios services configuration file, or replace the IP/name of the checked in file.

Generator script is located here:
https://github.com/fmadio/fmadio_scripts/blob/master/snmp/nagios_xi_service_gen.lua

Example usage is as follows:
$ ./nagios_xi_service_gen.lua --host 192.168.1.75 --fmad20-1u-16t --host --fmad20-1u-16t Generating for [fmadio20] at host [192.168.1.75] $ There will now be a nagios services configuration file in the local directory. In this case "192.168.1.75.cfg"

OR

Modify the checked in file, and rename all the IP address. Example services config file:
https://github.com/fmadio/fmadio_scripts/blob/master/snmp/192.168.1.75.cfg
$ ls -al total 116 drwxrwxr-x 2 aaron aaron 4096 11月 24 02:42 . drwxrwxr-x 4 aaron aaron 4096 11月 22 09:42 .. -rw-rw-r-- 1 aaron aaron 70709 11月 24 02:42 192.168.1.75.cfg -rwxrwxr-x 1 aaron aaron 23765 11月 22 09:42 FMADIO-MIB.txt -rwxrwxr-x 1 aaron aaron 9704 11月 24 02:42 nagios_xi_service_gen.lua 5) Copy services file to nagios server

On the nagios server there is a list of service configuration files.
[[email protected] services]# pwd /usr/local/nagios/etc/services [[email protected] services]# ls -al total 140 drwsrwsr-x 2 apache nagios 4096 Nov 20 14:27 . drwsrwsr-x 7 apache nagios 4096 Nov 17 22:48 .. -rw-rw-r-- 1 apache nagios 70709 Nov 19 00:33 192.168.1.75.cfg -rw-rw-r-- 1 apache nagios 3118 Nov 2 04:07 localhost.cfg [[email protected] services]# scp and overwrite the current IP.cfg file with the one generated above
$ scp 192.168.1.75.cfg [email protected]:/usr/local/nagios/etc/services/ [email protected]'s password: 192.168.1.75.cfg $
6) Reload the Nagios configuration

10g packet capture manual nagios setup 0
 
10g packet capture manual nagios setup 0
 
10g packet capture manual nagios setup 0
 
7) Nagios configuration is now complete.

Some examples shown below

10g packet capture manual nagios setup 0
 
10g packet capture manual nagios setup 0
 
10g packet capture manual nagios setup 0
 

Clock Synchronization

The FMADIO capture device has multiple time/clock synchronization methods. Please note, this setting is for synchronizing the local clock to World time (or a local Grand master). The hardware timestamp on every packet is always 1 nanosecond resolution.


Protocol Accuracy Description
PTPv2 < 100 nano seconds Precision Time Protocol Version 2
PTPv2 + PPS < 10 nano seconds Precision Time Protocol Version 2 With External PPS
GPS < 10 nano seconds Global Positioning System Time Synchronization (built in)
NTP ~ 1,000,000 nano seconds Network Time Protocol
NTP Manual ~ 1,000,000 nano seconds Manual Network Time Protocol Update

Depending on operational conditions and requirements, please choose the best time synchronization option for your device.



PTPv2 Time Sync

PTP Version 2 configuration provides the best accuracy that requires the least amount of setup/infrastructure. It does require a local PTP Grand master that is synchronized over ethernet. Typically we see 50-100ns accuracy using this method.

10g packet capture ptpv2


PTPv2 Time Sync + PPS Input

For applications requiring extreme timing accuracy this setup augments the PTPv2 synchronization with a 1PPS signal from the PTP GrandMaster/PTP Boundary switch. The additional PPS Input via the SMA coax cable provides 1 pulse per second timing accuracy to around 10nsec while the PTP protocol provides accurate date/time of day information.

10g packet capture ptpv2 with pps


GPS

Using the builtin GPS receiver with the included SMA cable provides excellent < 10nsec world time accuracy. It requires an active GPS antenna to be plugged in and connecting the GPS PPS Out port to the PPS In with the cable provided.


10g packet capture gps time synchronization


NTP

Network Time Protocol (NTP) is the most common and widely used time synchronization protocol. It requires an ethernet connection to the NTP time server, but the synchronization accuracy is not great, in 1-100 millisecond range.


10g packet capture ntp time synchronization


SSH Shell Settings

FMADIO devices run exclusively from pseudo-ROM where any changes on the file system between reboots is lost. This ROM approach provides consistency and system predictability making maintenance simpler.

One problem with this approach is shell customization becomes quite difficult. To allow small modifications in the shell environment when a user logs into the system it can run the shell script for each SSH session. Configuration file is:/mnt/store0/etc/fmadio.rc Please do not use this excessively, typically its used for setting ENV variables.

Example:
$ cat /mnt/store0/etc/fmadio.rc # local shell prompt configuration (ash) ran on at boot time export TEST="random test variable"

Operating Guide

Capture Start&Stop (Web)

Starting a capture manually can be performed using the Web interface or via Command Line Interface(CLI) from a remote machine. In most cases captures are started using the Scheduled Captures feature however there are many cases when starting an immediate capture is required.


Web Interface: Start Capture


From the dashboard Start by selecting the Capture menu option as highlighted in green below.



10g packet capture manual capture
 
 

From the capture page below we can see there is no capture running (highlighted in green below).

The steps to start a capture immediately:


  • 1) Enter a new capture name, in this example we enter "manual_capture" (highlighted in blue below)
  • 2) Start the capture by clicking on the Rec(ord) button (highlighted in red below)


10g packet capture manual capture
 

After clicking the REC button the web page will update as shown in the image below.


10g packet capture manual capture
 

We can see the capture status (highlighted in green above). This shows the capture is running, the capture name, how long it has been running and how many bytes/packets have been captured. Also (highlighted in blue above) on all web pages it shows a small REC icon to show the system is currently in a active capture state.

Web Interface: Stop capture


To stop any capture (both manual and scheduled) simply click on the STOP button as highlighted in green below.


10g packet capture manual capture stop
 

After stopping the web UI will look like the image below.


10g packet capture manual capture stop
 

Where the status shows no capture running, "Capture Running: false" (highlighted in green above). In addition, as no capture is active the recording toolbar icon visible on all web pages is now gone (blank space highlighted in blue above).

It is a clean and simple interface. If anything is unclear please contact us support @ fmad .io and we are happy to assist.

Capture Scheduling (Web)

Captures can be automatically scheduled to start/stop based on time and day of the week. This is best when monitoring specific time periods, e.g. Market hours or Broadcast time slots to conserves disk space. Scheduling capture is easy and straightforward as follows.

In this example we are capturing the time slot Monday - Firday from 7AM to 5PM as follows.


Step 1) Add a new row to the scheduling time table. Click on the Green Plus button as hilighted below


10g packet capture scheduling step 1
 

Step 2) A new row with a blank timeslot will be shown as hilighited in green below


10g packet capture scheduling step 1
 

Step 3) Enter a capture name prefix. The system automatically appends the suffix date/time _YYYYMMDD_HHMMSS for every capture started. For example the example below "test_capture" will generate capture names "test_capture_20160101_070000", "test_capture_20160102_070000", etc.


10g packet capture scheduling step 1
 

Step 4) Set the start and end times. Time is in HH:MM:SS 24 Hour format, based on the local time. In this example we`ve set from 07:00 -> 17:00 e.g. 7am to 5pm.


10g packet capture scheduling step 1
 

Step 5) Select the days of the week to capture. In this case we`ve selected Monday to Friday.


10g packet capture scheduling step 1
 

Scheduling captures is simple, multiple schedules are possible e.g One capture name for Mon-Fri and another capture name for Sat-Sun. To remove a scheduled capture click the X button to delete the row.

Capture 24/7 Always On

For many applications capturing 24/7 always on is a requirement. The following steps demonstrate how to setup 24/7 packet capture. In this mode if the Device is powered on it will be capturing, even after rebooting.


Step 1) Add a new row to the scheduling time table. Click on the Green Plus button as hilighted below


10g packet capture scheduling step 1
 

Step 2) A new row with a blank timeslot will be shown as hilighited in green below


10g packet capture scheduling step 1
 

Step 3) Enter a capture name prefix. The system automatically appends the suffix date/time _YYYYMMDD_HHMMSS for every capture started. For example the example below "always_capture" will generate capture names "always_capture_20160101_000000", "always_capture_20160102_000000", etc.


10g packet capture 24/7
 

Step 4) Set checkbox for 24/7 capture. Notice how time and day are now greyed out.


10g packet capture 24/7
 

The system will is now in always capture mode, if it is powered on it will be capturing even after rebooting. To stop 24/7 capture delete the row by clicking on the "X" button.

Capture Pre Filter

Filtering the packet stream before writing to storage has many applications. For example, dropping backup transfers, duplicate packet streams or slicing encrypted traffic for compliance reasons. Our FMADIO20 device offers 8 pre-capture filter rules to DROP, SLICE or ACCEPT packets before writing to storage. Please note, you can not mix DROP/SLICE rules with ACCEPT rules. e.g to use ACCPET all rules must be ACCEPT rules.

The following example drops all HTTPS data from writing to storage.

NOTE: Pre Capture Filter is only AVAILABLE on FMADIO20 devices


Step 1) Open the Advanced menu on the capture configuration page.



pre capture filtering
 
 

Step 2) Enable one of the Pre Filter rules.



pre capture filtering
 
 

Step 3) Enter the Pre-Filtering condition. In this case it is all HTTPS source traffic



pre capture filtering
 
 

Step 4) Enter the action to take. In this case dropping the packet entirely.



pre capture filtering
 
 

Step 5) Repeat steps for Destination Port for a Bi-Directional HTTPS filtering.



pre capture filtering
 
 

Only 8 simple rules are avaliable as filtering is must operate at full and sustained 20Gbps and 30Mpps. Internally there are 8 seprate mask and value compare`s on the first 128B of a packet. If you require custom filters please contact us.

Example Filters:

ipv4.src == 192.168.1.1IPv4 source filter single IP
ipv4.dst == 192.168.1.0/24 IPv4 dest filter /24 subnet
ipv4.proto == tcpIPv4 filter TCP traffic
ipv4.proto == udpIPv4 filter UDP traffic
ipv4.proto == 42IPv4 filter protocol 42
ipv6.src == 3ffe:507:0:1:200:86ff:fe05:80daIPv6 source filter single IP
ipv6.dst == 3ffe:507:0:1:200:86ff:fe05:80daIPv6 source filter single IP
mac.src == 00:01:02:03:04:05MAC Filter source address
mac.dst == 00:01:02:03:04:05MAC Filter dest address
mac.proto == 0x0806MAC Filter hex protcol number 0x0806 (ARP)
tcp.port.src == 80TCP source port filter 80 (HTTP)
tcp.port.dst == 80TCP dest port filter 80 (HTTP)
udp.port.src == 53UDP source port filter 53 (DNS)
udp.port.dst == 53UDP dest port filter 53 (DNS)

Capture Start&Stop (CLI)

Starting and stopping captures manually using the Command Line Interface (CLI) is extremely simple. It requires a correctly formatted URL request. In the examples below we use CURL but any program with HTTP functionality will work. Note that in the examples below the username is "user" and the password is "password", please replace with the correct information.


CLI Interface: Capture Status


Before starting and stopping captures, its helpful to check the current systems capture status. The following URL returns the current system status. curl "http://fmadio.probe.ip/sysmaster/status" For example when the packet sniffer is active and capturing data, the output will look like the below text
$ curl -u user:pass "http://192.168.11.75/sysmaster/status" uptime, 0D 3H 36M packets_received, 453468480 packets_dropped, 0 packets_errors, 0 packets_captured, 453468480 bytes_captured, 30835857408 bytes_pending, 4812701696 bytes_cache, 0 bytes_disk, 33742389248 capture_link, up capture_link_uptime, 0D 3H 36M capture_link_speed, 10000 capture_bytes, 31705286552 capture_packets, 466254210 capture_bps, 7726900224 capture_pps, 14203859 capture_name, manual_capture_cli capture_active, true And when the system is not capturing, it looks like the following below $ curl -u user:pass "http://192.168.11.75/sysmaster/status" uptime, 0D 3H 28M packets_received, 400000000 packets_dropped, 0 packets_errors, 0 packets_captured, 400000000 bytes_captured, 27200000000 bytes_pending, 0 bytes_cache, 0 bytes_disk, 33600831488 capture_link, up capture_link_uptime, 0D 3H 28M capture_link_speed, 10000 capture_bytes, 27200000000 capture_packets, 400000000 capture_bps, 0 capture_pps, 0 capture_name, none capture_active, false As you can see this is a simple format that's easy to parse, and excellent for compact monitoring scripts to use. For example: $ curl -s -u user:pass "http://192.168.11.75/sysmaster/status" | grep capture_active capture_active, true

CLI Interface: Start Capture


Starting captures from the CLI is also extremely easy, using the following URL format http://capture.sys.ip/sysmaster/capture_start?StreamName=enter_stream_name_here This will start a capture and return in JSON format the result of the request. The following example starts a capture with the name "cli_capture", then confirms its capture status $ curl -u user:pass "http://192.168.11.75/sysmaster/capture_start?StreamName=cli_capture" {"Status":true,"Str":"[Sat Jun 20 20:28:55 2015] successfully started capture [cli_capture]"} $ curl -u user:pass "http://192.168.11.75/sysmaster/status" uptime, 0D 0H 3M packets_received, 0 packets_dropped, 0 packets_errors, 0 packets_captured, 0 bytes_captured, 0 bytes_pending, 0 bytes_cache, 0 bytes_disk, 22325755904 capture_link, up capture_link_uptime, 0D 0H 3M capture_link_speed, 10000 capture_bytes, 0 capture_packets, 0 capture_bps, 0 capture_pps, 0 capture_name, cli_capture capture_active, true


CLI Interface: Stop Capture


Stopping captures via CLI is even simpler as it does not require a stream name. Use the following URL to stop any captures currently running. http://capture.sys.ip/sysmaster/capture_stop The following example shows the previous capture state, stopping the capture and verifying capture has stopped. $ curl -u user:pass "http://192.168.11.75/sysmaster/status" uptime, 0D 0H 9M packets_received, 101000000 packets_dropped, 0 packets_errors, 0 packets_captured, 101000000 bytes_captured, 6867999744 bytes_pending, 7142375424 bytes_cache, 0 bytes_disk, 23657971712 capture_link, up capture_link_uptime, 0D 0H 9M capture_link_speed, 10000 capture_bytes, 6868000000 capture_packets, 101000000 capture_bps, 0 capture_pps, 0 capture_name, cli_capture capture_active, true $ curl -u user:pass "http://192.168.11.75/sysmaster/capture_stop" {"Status":true,"Str":"[Sat Jun 20 20:39:17 2015] successfully stopped capture [cli_capture]"} $ curl -u user:pass "http://192.168.11.75/sysmaster/status" uptime, 0D 0H 11M packets_received, 101000000 packets_dropped, 0 packets_errors, 0 packets_captured, 101000000 bytes_captured, 6867999744 bytes_pending, 0 bytes_cache, 0 bytes_disk, 30809784320 capture_link, up capture_link_uptime, 0D 0H 11M capture_link_speed, 10000 capture_bytes, 6868000000 capture_packets, 101000000 capture_bps, 0 capture_pps, 0 capture_name, none capture_active, false Very simple and very easy so you spend time on the important tasks.



CLI Interface: Summary


Quick summary of CLI operations follows:

Operation URL
Start Capture http://capture.sys.ip/sysmaster/capture_start?StreamName=insert_stream_name_here
Stop Capture http://capture.sys.ip/sysmaster/capture_stop
Capture Status http://capture.sys.ip/sysmaster/status

PCAP Downloads

PCAP Download (Web)

Simple and intuitive downloading of PCAP`s is an important design goal of the FMADIO capture system. The FMADIO capture system provides simultaneous multiple views of the data for example 1 second PCAP split, 1 hour PCAP splits, 1GB splits and many more. Of course fetching the entire PCAP as a single file is also supported.

The list of supported PCAP split options is as follows.


  • Single file
  • Split 1 Second
  • Split 10 Second
  • Split 1 Minute
  • Split 10 Minute
  • Split 15 Minute
  • Split 1 Hour
  • Split 1 MB size
  • Split 10 MB size
  • Split 100 MB size
  • Split 1 GB size
  • Split 10 GB size
  • Split 100 GB size
  • Split 1 TB size

All views/splits are available for every capture simultaneously

You can view, download or analyze captures with a 1 hour split but also 1 second split without any configuration changes. Our highly optimized software enables splitting captures in multiple ways enabling efficient network trouble shooting.


Web: Download PCAP


Start by accessing the File menu as highlighted in green below.

10g packet capture manual PCAP download
 
After clicking on the Files menu, it shows all captures captures currently on the system. In the example below we want the capture named "manual_capture_cli_20150620_1836", the icon highlighted in blue is a quick link to download the entire capture as a single PCAP file.

10g packet capture manual PCAP download select stream
 
When clicking on the link highlighted in green above, the system displays all splits and views of the capture(shown in the image below). As all views of the capture are displayed, it enables you to select the most appropriate view for the task at hand. In this case we will select 1 second split as highlighted in green below.

10g packet capture manual PCAP download split 1sec
 
After clicking on the 1 second view from the above image, the full list of per second splits is seen in the screenshot below.

10g packet capture manual PCAP download split 1sec download
 
Clicking on the PCAP icon (highlighted in green above) starts a download of the file to local disk. After the download has completed, the file can be opened up in Wireshark or any other PCAP processing utility. In the example below Wireshark has loaded the requested file.

10g packet capture manual PCAP download split 1sec download wireshark
 
This is a short example of how PCAP files are extracted from the system. See the Advanced section for details on other approaches.

PCAP Download (CLI)

Downloading via a Web interface is intuitive and simple, but is not ideal when integrating with existing scripts and infrastructure. In this section we demonstrate how to use HTTP requests and a simple Command Line Interface (CLI) to find and download the PCAP`s you need.

CLI: Download PCAP


Start by listing all streams on the device in a comma delimited text file using the following URL format
http://192.168.11.75/plain/list For example.
$ curl -u user:pass "http://192.168.11.75/plain/list" Filename , Size Bytes , Packet Count , Date ,Single PCAP Link,File Link, cli_capture_20150620_2028 , 8484028416, 101000000, Sat Jun 20 20:28:56 2015,/pcap/single?StreamName=cli_capture_20150620_2028&,/en.files.html?Fn=view&StreamName=cli_capture_20150620_2028&, manual_capture_cli_20150620_1836 , 84000112640, 1000000000, Sat Jun 20 18:36:12 2015,/pcap/single?StreamName=manual_capture_cli_20150620_1836&,/en.files.html?Fn=view&StreamName=manual_capture_cli_20150620_1836&, remote_split_1434790902645_20150620_1802 , 8400142336, 100000000, Sat Jun 20 18:02:06 2015,/pcap/single?StreamName=remote_split_1434790902645_20150620_1802&,/en.files.html?Fn=view&StreamName=remote_split_1434790902645_20150620_1802&, remote_split_1434790250968_20150620_1751 , 8400142336, 100000000, Sat Jun 20 17:51:14 2015,/pcap/single?StreamName=remote_split_1434790250968_20150620_1751&,/en.files.html?Fn=view&StreamName=remote_split_1434790250968_20150620_1751&, manual_capture_20150620_1730 , 262144, 0, Sat Jun 20 17:30:13 2015,/pcap/single?StreamName=manual_capture_20150620_1730&,/en.files.html?Fn=view&StreamName=manual_capture_20150620_1730&, remote_split_1434781447634_20150620_1524 , 8400142336, 100000000, Sat Jun 20 15:24:31 2015,/pcap/single?StreamName=remote_split_1434781447634_20150620_1524&,/en.files.html?Fn=view&StreamName=remote_split_1434781447634_20150620_1524&, remote_split_1434780537092_20150620_1509 , 8400142336, 100000000, Sat Jun 20 15:09:21 2015,/pcap/single?StreamName=remote_split_1434780537092_20150620_1509&,/en.files.html?Fn=view&StreamName=remote_split_1434780537092_20150620_1509&, remote_capture_reboot_1434777911130445056_20150620_1425 , 8399880192, 99998575, Sat Jun 20 14:25:35 2015,/pcap/single?StreamName=remote_capture_reboot_1434777911130445056_20150620_1425&,/en.files.html?Fn=view&StreamName=remote_capture_reboot_1434777911130445056_20150620_1425&, remote_capture_reboot_1434777685258077952_20150620_1421 , 8399880192, 99998575, Sat Jun 20 14:21:49 2015,/pcap/single?StreamName=remote_capture_reboot_1434777685258077952_20150620_1421&,/en.files.html?Fn=view&StreamName=remote_capture_reboot_1434777685258077952_20150620_1421&, remote_capture_reboot_1434777459172079104_20150620_1418 , 8399880192, 99998575, Sat Jun 20 14:18:03 2015,/pcap/single?StreamName=remote_capture_reboot_1434777459172079104_20150620_1418&,/en.files.html?Fn=view&StreamName=remote_capture_reboot_1434777459172079104_20150620_1418&, remote_capture_reboot_1434777233614947072_20150620_1414 , 8399880192, 99998575, Sat Jun 20 14:14:17 2015,/pcap/single?StreamName=remote_capture_reboot_1434777233614947072_20150620_1414&,/en.files.html?Fn=view&StreamName=remote_capture_reboot_1434777233614947072_20150620_1414&, remote_capture_reboot_1434777007262298880_20150620_1410 , 8399880192, 99998575, Sat Jun 20 14:10:31 2015,/pcap/single?StreamName=remote_capture_reboot_1434777007262298880_20150620_1410&,/en.files.html?Fn=view&StreamName=remote_capture_reboot_1434777007262298880_20150620_1410&, remote_capture_reboot_1434776760119515904_20150620_1406 , 8399880192, 99998575, Sat Jun 20 14:06:24 2015,/pcap/single?StreamName=remote_capture_reboot_1434776760119515904_20150620_1406&,/en.files.html?Fn=view&StreamName=remote_capture_reboot_1434776760119515904_20150620_1406&, remote_capture_reboot_1434776535094639104_20150620_1402 , 8399880192, 99998575, Sat Jun 20 14:02:39 2015,/pcap/single?StreamName=remote_capture_reboot_1434776535094639104_20150620_1402&,/en.files.html?Fn=view&StreamName=remote_capture_reboot_1434776535094639104_20150620_1402&, remote_capture_reboot_1434776309877481984_20150620_1358 , 8399880192, 99998575, Sat Jun 20 13:58:54 2015,/pcap/single?StreamName=remote_capture_reboot_1434776309877481984_20150620_1358&,/en.files.html?Fn=view&StreamName=remote_capture_reboot_1434776309877481984_20150620_1358&, remote_capture_reboot_1434776084655464960_20150620_1355 , 8399880192, 99998575, Sat Jun 20 13:55:09 2015,/pcap/single?StreamName=remote_capture_reboot_1434776084655464960_20150620_1355&,/en.files.html?Fn=view&StreamName=remote_capture_reboot_1434776084655464960_20150620_1355&, remote_capture_reboot_1434775857769832960_20150620_1351 , 8399880192, 99998575, Sat Jun 20 13:51:22 2015,/pcap/single?StreamName=remote_capture_reboot_1434775857769832960_20150620_1351&,/en.files.html?Fn=view&StreamName=remote_capture_reboot_1434775857769832960_20150620_1351&, remote_split_1434775647869_20150620_1347 , 8400142336, 100000000, Sat Jun 20 13:47:52 2015,/pcap/single?StreamName=remote_split_1434775647869_20150620_1347&,/en.files.html?Fn=view&StreamName=remote_split_1434775647869_20150620_1347&, remote_reboot1434773930619883008_20150620_1319 , 8400142336, 100000000, Sat Jun 20 13:19:15 2015,/pcap/single?StreamName=remote_reboot1434773930619883008_20150620_1319&,/en.files.html?Fn=view&StreamName=remote_reboot1434773930619883008_20150620_1319&, remote_reboot1434773735263832064_20150620_1315 , 8400142336, 100000000, Sat Jun 20 13:15:59 2015,/pcap/single?StreamName=remote_reboot1434773735263832064_20150620_1315&,/en.files.html?Fn=view&StreamName=remote_reboot1434773735263832064_20150620_1315&, remote_reboot1434773540472738048_20150620_1312 , 8400142336, 100000000, Sat Jun 20 13:12:44 2015,/pcap/single?StreamName=remote_reboot1434773540472738048_20150620_1312&,/en.files.html?Fn=view&StreamName=remote_reboot1434773540472738048_20150620_1312&, remote_reboot1434773345743976960_20150620_1309 , 8400142336, 100000000, Sat Jun 20 13:09:30 2015,/pcap/single?StreamName=remote_reboot1434773345743976960_20150620_1309&,/en.files.html?Fn=view&StreamName=remote_reboot1434773345743976960_20150620_1309&, remote_reboot1434773148835964928_20150620_1306 , 8400142336, 100000000, Sat Jun 20 13:06:13 2015,/pcap/single?StreamName=remote_reboot1434773148835964928_20150620_1306&,/en.files.html?Fn=view&StreamName=remote_reboot1434773148835964928_20150620_1306&, remote_reboot1434772955000894976_20150620_1302 , 8400142336, 100000000, Sat Jun 20 13:02:59 2015,/pcap/single?StreamName=remote_reboot1434772955000894976_20150620_1302&,/en.files.html?Fn=view&StreamName=remote_reboot1434772955000894976_20150620_1302&, remote_reboot1434772759495929088_20150620_1259 , 8400142336, 100000000, Sat Jun 20 12:59:44 2015,/pcap/single?StreamName=remote_reboot1434772759495929088_20150620_1259&,/en.files.html?Fn=view&StreamName=remote_reboot1434772759495929088_20150620_1259&, remote_reboot1434772564984750080_20150620_1256 , 8400142336, 100000000, Sat Jun 20 12:56:29 2015,/pcap/single?StreamName=remote_reboot1434772564984750080_20150620_1256&,/en.files.html?Fn=view&StreamName=remote_reboot1434772564984750080_20150620_1256&, remote_reboot1434772368323953920_20150620_1253 , 8400142336, 100000000, Sat Jun 20 12:53:12 2015,/pcap/single?StreamName=remote_reboot1434772368323953920_20150620_1253&,/en.files.html?Fn=view&StreamName=remote_reboot1434772368323953920_20150620_1253&, remote_reboot1434772173499830016_20150620_1249 , 8400142336, 100000000, Sat Jun 20 12:49:58 2015,/pcap/single?StreamName=remote_reboot1434772173499830016_20150620_1249&,/en.files.html?Fn=view&StreamName=remote_reboot1434772173499830016_20150620_1249&, remote_reboot1434771978707971072_20150620_1246 , 8400142336, 100000000, Sat Jun 20 12:46:43 2015,/pcap/single?StreamName=remote_reboot1434771978707971072_20150620_1246&,/en.files.html?Fn=view&StreamName=remote_reboot1434771978707971072_20150620_1246&,
Above is the example output that's clear and simple to parse. In this case we will select the stream "manual_capture_cli_20150620_1836" to download as a single file.


CLI: PCAP Download Single File

To download the capture "manual_capture_cli_20150620_1836" as a single PCAP use the following URL format curl -u user:pass http://192.168.11.75/pcap/single?StreamName=InsertName > /capture/todays.pcap For example to download the capture "manual_capture_cli_20150620_1836" in its entirety. $ curl -u user:pass "http://192.168.11.75/pcap/single?StreamName=manual_capture_cli_20150620_1836" > /capture/todays.pcap % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 60 78.2G 60 47.3G 0 0 1010M 0 0:01:19 0:00:47 0:00:32 1024M $ Notice the excellent download speed, that approaches the full 10Gbit management port capacity.


CLI: PCAP Split Download

To view all the split options for a capture use the following URL pattern curl -u user:pass http://192.168.11.75/plain/view?StreamName=InsertName For example the following request below shows all split options for the capture named "manual_capture_cli_20150620_1836" $ curl -u user:pass "http://192.168.11.75/plain/view?StreamName=manual_capture_cli_20150620_1836" SplitMode , Link split_1sec , /plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_1sec split_10sec , /plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_10sec split_1min , /plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_1min split_10min , /plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_10min split_15min , /plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_15min split_1hour , /plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_1hour split_1GB , /plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_1GB split_10GB , /plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_10GB split_100GB , /plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_100GB split_1TB , /plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_1TB Where the first column is the split operation, and the second column is the URL to list the split files. After deciding on the split mode, the following URL pattern is used to list all files. curl -u user:pass http://192.168.11.75/plain/split?StreamName=InsertName&StreamView=InsertView For example, extracting a list of 1 second splits from the capture named "manual_capture_cli_20150620_1836" $ curl -u user:pass "http://192.168.11.75/plain/split?StreamName=manual_capture_cli_20150620_1836&StreamView=split_1sec" Name , Size Bytes, Packet Count, URL 20150620_18:36:47.780.373.248 , 1193017344, 14202589, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793007780373366ULL&Stop=1434793008780383914ULL 20150620_18:36:48.780.384.000 , 1193279488, 14205708, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793008780383914ULL&Stop=1434793009780514262ULL 20150620_18:36:49.780.514.304 , 1193017344, 14202588, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793009780514262ULL&Stop=1434793010780424945ULL 20150620_18:36:50.780.424.960 , 1193279488, 14205708, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793010780424945ULL&Stop=1434793011780555292ULL 20150620_18:36:51.780.555.264 , 1193017344, 14202587, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793011780555292ULL&Stop=1434793012780465900ULL 20150620_18:36:52.780.465.920 , 1193017344, 14202588, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793012780465900ULL&Stop=1434793013780376583ULL 20150620_18:36:53.780.376.576 , 1193279488, 14205708, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793013780376583ULL&Stop=1434793014780506931ULL 20150620_18:36:54.780.506.880 , 1193017344, 14202587, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793014780506931ULL&Stop=1434793015780417554ULL 20150620_18:36:55.780.417.536 , 1193279488, 14205708, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793015780417554ULL&Stop=1434793016780547901ULL 20150620_18:36:56.780.547.840 , 1193017344, 14202588, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793016780547901ULL&Stop=1434793017780458591ULL 20150620_18:36:57.780.458.496 , 1193279488, 14205708, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793017780458591ULL&Stop=1434793018780588947ULL 20150620_18:36:58.780.589.056 , 1193017344, 14202587, /pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793018780588947ULL&Stop=1434793019780499570ULL . . . The above shows all split filenames of capture ""manual_capture_cli_20150620_1836" at 1 second split interval. To download a specific split, for example the 1 second capture at "20150620_18:36:58" use the link obtained above and issue the CURL get command shown below. In this example we`re piping the output into tcpdump via stdin. $ curl -u user:pass "http://192.168.11.75/pcap/splittime?StreamName=manual_capture_cli_20150620_1836&Start=1434793018780588947ULL&Stop=1434793019780499570ULL" | ./tcpdump_ns -r - -nn % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 24 0 24 0 0 2 0 --:--:-- 0:00:08 --:--:-- 0reading from file -, link-type EN10MB (Ethernet) 18:36:58.780.589.081 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 76, rcv seq 18, Flags [Command], length 54 18:36:58.780.589.163 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 82, rcv seq 18, Flags [Command], length 54 18:36:58.780.589.222 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 88, rcv seq 18, Flags [Command], length 54 18:36:58.780.589.297 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 94, rcv seq 18, Flags [Command], length 54 18:36:58.780.589.364 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 100, rcv seq 18, Flags [Command], length 54 18:36:58.780.589.438 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 106, rcv seq 18, Flags [Command], length 54 18:36:58.780.589.506 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 112, rcv seq 18, Flags [Command], length 54 18:36:58.780.589.573 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 118, rcv seq 18, Flags [Command], length 54 18:36:58.780.589.655 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 124, rcv seq 18, Flags [Command], length 54 Integration with your existing scripts and infrastructure is achieved using simple HTTP/HTTPS requests that are clean and easy to understand.


CLI Download: Summary


Quick summary of CLI download operations:

Operation URL
List Captures http://capture.sys.ip/plain/list
Single file download http://capture.sys.ip/pcap/single?StreamName=InsertName
List Capture`s various split`s http://capture.sys.ip/plain/view?StreamName=InsertName
List Capture`s specific split file list http://capture.sys.ip/plain/split?StreamName=InsertName?StreamView=InsertView

PCAP Analysis

Packet Filtering

For post capture analysis, Filter blocks and Filter Expressions enable complex and almost arbitrary filtering capabilities. We separate filtering into simple per packet "Filter Blocks", for example Is the packet UDP? which are combined into more complicated "Filter Expressions". This enables you to chain and build complex filter expressions based on your requirements.


Simple Filter

Starting with a very simple filter, "Extract all TCP packets" we build in the following way.

Step 1) Enter the filter expression "ipv4.proto == tcp" in the filter block highlighted in green below. After pressing enter the display will look like the image below.


10g packet capture manual capture stop


Step 2) Rename the Filter Expression to "all_tcp" by clicking on and editing the text in the area highlighted in green below. This improves readability but for simple filters it is excessive and not required.


10g packet capture manual capture stop


Step 3) Update the filter expression by clicking and editing the area highlighted in green below to reference the newly named "all_tcp" Filter Block. For simple filters the default values of "block0" can be used.


10g packet capture manual capture stop


Step 4) Run packet Analysis or Download to PCAP by selecting on the appropriate icons as shown below. (shown below is green to run analysis, blue to download as PCAP)

10g packet capture manual capture stop


Below is a quick list of example filters currently supported. If you require additional filters please let us know.

ether.proto == ipv4 select all IPv4 traffic
ether.addr == 00:ba:be: select all ethernet mac address starting with wild card 00:ba:be:*:*:*
ether.addr == 00:ba:be:11:22:33 select all packets with exact mac address (src or dest) of 00:ba:be:11:22:33
ether.addr == 00:* select all packets with wildcard mac address (src or dest)
ether.fcs == fail select all packets with invalid ethernet frame FCS
ipv4.addr == 192.168.1.1 select all ipv4 packets with address of 192.168.1.1
ipv4.addr == 192.168.1.0/24 select all ipv4 packets on subnet 192.168.1.0/24
ipv4.addr.src == 192.168.1.0/24 select all ipv4 packets whose source address is on subnet 192.168.1.0/24
ipv4.proto == tcp select all ipv4 TCP packets
ipv4.proto == udp select all ipv4 UDP packets
ipv4.proto == icmp select all ipv4 ICMP packets
ipv4.proto == igmp select all ipv4 IGMP packets
tcp.port == 1000 select all tcp traffic with port number 1000
tcp.port == 1000-2000 select all tcp traffic with port number range from 1000 to 2000
tcp.port.src == 1000 select all tcp traffic with source port number 1000
udp.port == 3000 select all udp traffic with port number 3000
udp.port == 3000-4000 select all udp traffic with port number range from 3000 to 4000
udp.port.src == 3000 select all udp traffic with source port number 3000
frame.time == 20141225_08:00:00.100.200.300 select all packets after the date 2014 12(Dec) 25th from 8:00:00.100.200.300
frame.time == 20141225_08:00:00.100.200.300-20141225_17:00:00.400.500.600 select all packets after the date 2014 12(Dec) 25th from 8:00:00.100.200.300 but before 2014 12(Dec) 25th at 17:00:00.400.500.600
frame.time == 07:00:00 select all packets after todays date at 07:00:00am
frame.size <= 128 select all packets less than or equal to 128 bytes in length
frame.size >= 1024 select all packets greater than or equal to 1024 bytes in length
frame.size == 64 select all packets equal to 64 bytes in length
vlan.id == 1234 select vlan tag id 1234



Complex Filters

Complex filters are possible by combining multiple Filter Blocks using Filter Expressions. The system parses simple boolean logic expression which references Filter blocks specified above.

Example Filter (Simple):

Simple One block expression to match all ethernet MAC address(effectively no filtering). Edit the filter Block name (in green below) to "default" and replacing the Filter Expression (in blue below) to "default".

10g packet capture manual capture stop


Example Filter (UDP and TCP traffic):

This is a 2 block filter, where the first block "all_udp" (in Green below) selects all UDP packets. The second block "all_tcp" (in green below) selects all TCP packets. And the Filter expression (in blue below) selects "all_udp" packets OR "all_tcp" packets. Net result is filtering for either TCP or UDP packets.

10g packet capture manual capture stop


Example Filter (UDP Port 5000 and TCP traffic):

This is a 3 block filter, where the first block "all_udp" (in Green below) selects all UDP packets. The second block "all_tcp" (in green below) selects all TCP packets. And the 3rd block "all_udp_port_5000" selects UDP packets on port 5000.

The Filter expression (in blue below) is some what verbose, as it selects TCP packets (but not UDP) "((!all_udp) & (all_tcp))", technically just all_tcp is sufficient but is expanded for demonstration purposes. OR all UDP packets on port 5000 "all_udp_port_5000". Net result is all TCP and UDP port 5000 packets are selected.

10g packet capture manual capture stop

Trouble shooting

System Log Generation

The first step for any problem resolution is generating detailed log files for analysis to understand the exact nature of the problem. Our system automatically generates logfile information using the following steps.

Step 1)

Select the Tools menu hilighted in green below.


10g packet capture crash dump debug

Step 2)

Start System Log generation, by clicking on the icon hilighted in green below.


10g packet capture crash dump debug

Step 3)

Logfile generation starts with status information shown in area hilighted in green. Depending on the size of logfile this may take from 1 - 15 minutes to complete.


10g packet capture crash dump debug

Step 4)

When completed the status will change as hilighted in green below.


10g packet capture crash dump debug

Step 5)

You can now download the report via the icon hilighted in Green. An example downloaded log file is shown in blue below. After download, transfer to us for further analysis.


10g packet capture crash dump debug



Backup a)

If for some reason the Web GUI has failed the log files are located in this directory [email protected]:/mnt/store0/upload$ ls -al total 2874860 drwxr-xr-x 2 root root 36864 Dec 29 12:43 ./ drwxr-xr-x 169 fmadio staff 12288 Dec 29 12:27 ../ -rw-r--r-- 1 root root 145024 Dec 29 12:27 filelist lrwxrwxrwx 1 root root 70 Dec 29 12:43 report.tar.gz -> /mnt/store0/upload/report_fmadio10_002590FC883C_20151229_122658.tar.gz -rw-r--r-- 1 root root 1285506376 Dec 29 12:43 report.tar.gz.asc -rw-r--r-- 1 root root 949296603 Dec 29 12:43 report_fmadio10_002590FC883C_20151229_122658.tar.gz
The file named filelist is the list of files contained in the logfile. There is a standard tarball and also encrypted tarball. Packet data is included in the log files and please unpack and check the contents complies with your companies security policy.




Backup b)

If logfile generation failed, you can manually invoke system log generation via command line, as shown below [email protected]:/mnt/store0/upload$ sudo /opt/fmadio/bin/syslog_report.lua fmad fmadlua Dec 22 2015 calibrating... 0 : 00000000d09dad48 3.5000 cycles/nsec Cycles/Sec 3499994440.0000 Std: 0cycle std( 0.00000000) loading filename [/opt/fmadio/bin/syslog_report.lua] Cmd [/opt/fmadio/bin/system_dump.lua > /mnt/store0/log/system_dump_20151229_132103] loading filename [/opt/fmadio/bin/system_dump.lua] [ iosched_direct.stdouterr_20151229] 1283855 1 MB [ iosched_direct_20151229_1205] 1365723 2 MB [ monitor_gps_20151229_1205] 9834318 12 MB [ monitor_memory_20151229_1205] 809724 13 MB [ monitor_nic_20151229_1205] 1179945 14 MB [ statusqueue_20151229_132103.tar.gz] 40916 14 MB [ stream_capture_sf20_20151229_1205] 288414 14 MB [ monitor_cpu_20151229_1205] 642415 15 MB [ scheduler_20151229_1205] 404614 15 MB [ sfptp_stats] 3276884 19 MB [ stream_writeback.stdouterr_20151229] 973105 20 MB [ stream_writeback_20151229_1205] 1054488 21 MB [ system_dump_20151229_132103] 1089180 22 MB [ monitor_ptp.lua.stdouterr_20151229] 22197 22 MB [ monitor_ptp_20151229_1205] 676222 23 MB [ analytics.lua.stdouterr_20151229] 30954 23 MB . . . . . .

IPMI Port Access

The Intelligent Platform Management Interface (IPMI) is designed as an out-of-band communication channel, used when normal connectivity with the server has been compromised. If your unable to connect with the system using SSH or HTTP(s) this out-of-band management interface can log into the FMADIO system via the serial port over ethernet.

From the rear port view of the FMADIO10/20 packet capture system, there is are dedicated RJ45 sockets for the IPMI interface highlighted in green. These are connected to your out-of-band management infrastructure, ideally on completely separate subnets and switch`s.


FMADIO10
10g packet capture ipmi port
FMADIO20
10g packet capture ipmi port

       

The interfaces support ICMP ping, HTTP and SSH protocols IP Address can be configured via the system BIOS or preferable using the FMADIO web interface as highlighted in green below. In this example IPMI port has a static IPv4 Address of 192.168.11.83 on the 192.168.11.0/24 subnet.

10g packet capture ipmi config



     

Serial Port Login

Serial port B on the system is connected to the IPMI interface. This allows us to login into the system on the Serial port and is very helpful if the regular network is down for some reason. The procedure is as follows: 16:15:47$ ssh [email protected] [email protected]'s password: ATEN SMASH-CLP System Management Shell, version 1.05 Copyright (c) 2008-2009 by ATEN International CO., Ltd. All Rights Reserved -> cd system1 /system1 -> cd sol1 /system1/sol1 -> start /system1/sol1 press , , and then to terminate session (press the keys in sequence, one after the other) Core Linux fmadio10-055 login: fmadio Password: _____ .___.__ 10G _/ ____\_____ _____ __| _/|__| ____ \ __\/ \ \__ \ / __ | | | / _ \ | | | Y Y \ / __ \_/ /_/ | | |( <_> ) |__| |__|_| /(____ /\____ | |__| \____/ \/ \/ \/ ============================================ -+ no user serviceable parts inside +- [email protected]:~$ And you now have full access to the system, even when the regular network is down.



Power Cycle

Power cycling / Power Reset the system can be done using the IPMI interface as shown below.
$ ssh [email protected] [email protected]'s password: ATEN SMASH-CLP System Management Shell, version 1.05 Copyright (c) 2008-2009 by ATEN International CO., Ltd. All Rights Reserved -> cd system1 /system1 -> cd pwrmgtsvc1 /system1/pwrmgtsvc1 -> reset /system1/pwrmgtsvc1 reset done... The system will then power down and reboot, it may take a 1 minute for regular SSH and HTTP access to be restored.



Power Off

To power off the system follow the following commands:
$ ssh [email protected] [email protected]'s password: ATEN SMASH-CLP System Management Shell, version 1.05 Copyright (c) 2008-2009 by ATEN International CO., Ltd. All Rights Reserved -> cd system1 /system1 -> cd pwrmgtsvc1 /system1/pwrmgtsvc1 -> stop /system1/pwrmgtsvc1 stop done... The system will then power down completely.



Power On

To power On the system follow the following commands:
$ ssh [email protected] [email protected]'s password: ATEN SMASH-CLP System Management Shell, version 1.05 Copyright (c) 2008-2009 by ATEN International CO., Ltd. All Rights Reserved -> cd system1 /system1 -> cd pwrmgtsvc1 /system1/pwrmgtsvc1 -> start /system1/pwrmgtsvc1 start done... The system will then power up.

Summary

FMADIO10 and FMADIO20 devices are cost effective line rate 10Gbe and 20Gbe packet capture systems. These systems are designed for simple and easy operation with excellent integration into your existing environment. If you have any further questions or requirements please contact us at any time.