Child pages
  • Tracking Flow Volume and Import Performance
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Flow Volume

opflow_stats.pl calculates how many flows per second are being received by the server.  To view this info just run it

/usr/local/opmantek/bin/opflow_stats.pl

Output: 

..............................................
Average Flows Per Second: 762
Average Flows per Cycle: 34
Flows Inserted per Cycle: 33
Conversations per Cycle: 23
Conversations Compression: 69%
Average LoadFlow: 0.0237999999999999
Average FlowReader: 0.0104999999999999
Average MvKill: 0.0101249999999999

In this example we are receiving 762 flows a second.  

Average flows per cycle is how many flows are received in a 2 minute cycle.  In this case it's saying 34, you may immediately think "this makes no sense" and you are correct.  The system this info was taken from had been saving up flows (opflowd was not running) and then it inserted them all at once followed by very few flows being received after.  This is a good representation of inconsistent flow records making the numbers look incorrect. 

Import Performance

In addition to saving the flows to disk, opFlow also needs to parse and import the flows into the database.  To get an idea of how many flows per second (fps) opflowd is importing into mongo run this command: 

tail -500 /usr/local/opmantek/logs/opflow.log | grep "STATS Flows"

Output will have lines like this:

10-Mar-2013 17:20:06,opflowd.pl::runFlows#338<br>opFlowd: STATS Flows: FPS=1964 Flows=10682 FlowInserts=10681 ConvInserts=7300 LoadFlow=5.44 FlowReader=0.25 MvKill=0.00 
10-Mar-2013 17:24:00,opflowd.pl::runFlows#338<br>opFlowd: STATS Flows: FPS=1050 Flows=21 FlowInserts=20 ConvInserts=16 LoadFlow=0.02 FlowReader=0.01 MvKill=0.01

As you can see the system this info was taken from is very erratic (as mentioned above).

By default opFlowd runs every 2 minutes, the output tells us how many FPS (flows per second) it is able to import, how many inserts were made as well has how many Converstation Inserts were made.  

Conversations are a group of flows between two endpoints that have been summarised over a given period defined in the opFlow.nmis config file, this is done to reduce the amount of data that is saved and therefor use less space.  This will only happen if they are enabled, and you will see this data displayed (instead of raw flows) if display is enabled:

  'opflow_summarisation_interval' => '60',
'opflow_summarisation_enabled' => 'true',
'opflow_summarisation_display' => 'true',

If you wish to only use summarised data (and not raw data, most likely to save space) you will want to turn off raw flow data:

  'opflow_keep_raw_flows' => 'true',
  • No labels