Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: reworked for version 3.x

Table of Contents

Introduction

Historically opReports has relied solely on being run on demand via the GUI or command line , (or periodically with cron; However with version 2.6.1 ). In version 3.0 we've added another feature that extends these capabilities: the opReports Scheduler. The scheduler is meant to act a bit like a combination of "cron" and "at" (a venerable but not especially well known Unix tool, companion of cron) but fine-tuned and optimized   for opReports.

The  The basic idea of the scheduler is that you can define control the scheduler by defining opReports "jobs" to be taken care of (once, at a particular time in the future, and such jobs can consist of actions to be taken both at the commencement and the termination of the reporting job period.

Limitations

As of version 2.6.1 this feature is to be considered a bit experimental and some of the details will likely have to change during the next few releases.

  • Currently only the scheduling of Interface Utilisation Reports is supported.
  • Scheduled jobs are currently one-offs, but support for jobs that run periodically is planned.
  • The NMIS remote control feature is very much geared towards Interface Utilisation Reports at this time.
  • At this time the only action to take at the beginning of a job is enabling NMIS and setting the listed interfaces and nodes for collection; This capability set will definitely be extended.

Scheduling Reports

or recurring with specific frequencies and start/end times/dates).

How to define Report Schedules

Each scheduled job is defined by a separate JSON file. The scheduler expects Scheduled jobs are controlled by a JSON file that defines the job parameters. The scheduler looks for these files in /usr/local/opmantekomk/conf/schedule , (but this is configurable (- see <omk_schedule> in conf/opReportsopCommon.nmis). As of version 3.0 schedule files must be managed using an external editor, but support for editing schedules from the GUI is already planned.

All job files must be called <something>.json, and only letters, digits, dot, underscore and hyphen are allowed in the file name. All other files are silently ignored. ( You can thus disable a job by naming it something like !disabled_job.json. opReports ships with a number of example schedule files (all set to be inactive) and a brief README file in the conf/schedule directory.

A job file will normally must consist of one "hash" datastructure, with the following properties (list subject to change/amendment)::

 

PropertyValuesExampleDescription

type

"util" (fixed!)"
"wan"
The report type to generate. This MUST be present to advise the scheduler what to do. See Limitations aboveRequired. See the opReports Report Descriptions for a list of available types.
active"true" or "false"Whether this schedule is active or not. Only active schedules can create new reports, inactive schedules are consulted only for pruning of old report data. If active is not specified, then the schedule is inactive.
description"some text"A free-form description of the scheduled job. Optional.
frequency"daily", "weekly", "monthly" or "yearly"Not present for one-off reports.
start"1-14-2015 20:00:00",
"mon 10:30", various others depending on the frequency

The date + time the report collection is to start at. This uses the same format  as elsewhere in opReports (e.g. "4-jan-2014 17:23:06")Required.
See the Formats for Report Periods and Frequencies page for details.

end The date + time the report collection is to end at. Same format as start.
from13 or any integer between 0 and  24Start of Business Hours reporting. Optional. Default: 0. See the Formats for Report Periods and Frequencies page for details.
to End of Business Hours reporting. Optional. Default: 24.
node_intf_list/some/where/some.fileThe file describing the nodes and interfaces to collect. See section below for format.
util_threshold80

The desired interface utilisation threshold value (in percent), default: 80 (percent).

util_threshold_mincount1

The desired minimum number of threshold exceedences for flagging the interface as over-limit, default: 1.

format

 

either a single "html"
", "csv" or "xlsx", or a list like [ "csv", "xlsx" ]
The format the  formats the report should be saved in.
targetfileoutputdir/some/where/new.fileWhere the report should be saved. This has to be a full path pointing to a directory.
It is suggested that you either use a directory under /var (or /data if you use the Opmantek Virtual Appliance), or /usr/local/omk/var/reports. Every report schedule can use a different output directory.
naming"simple" or "precise"What naming scheme should be used for the output files. Default is "simple". Mainly important if you plan to manually work with saved report files.
at most one of node_list, node_regexp,  node_group, node_intf_list or node_intf_type_list See How to select Nodes (and Interfaces) for details.
keep_for47How many days to keep an old report instance. If set to zero, the report is not expired. If not set, then the defaults for the report type are applied  (configuration entry "default_report_keep_for").
options

{ "title" : "My Custom ReportTitle" }

 

Optional settings. Some are specific to particular report types. See the section on Optional Settings below and the file this points to will  be overwritten when the report is generated.
control_nmis

"true" or "false"

Whether NMIS should be "remote controlled" or left in peace. See section on Remote Controlling NMIS below. Only relevant for one-off reports.
nmis_options"mthread=true maxthreads=15"Options to be given to nmis.pl type=update. Optional, and ignored relevant only  if control_nmis is falsetrue.

Besides these user-definable properties, the scheduler will also manage manages certain other properties others for internal use, and you should not modify these! (Currently this includes these include the properties "activeuuid", "in_progress", "completed" and "completedall properties named "actual_<something>".)

The scheduler checks the start and end dates and determines whether any actions need to be taken; if the start time has been reached, then the job is marked as active (and optionally NMIS is remote controlled). When the end time has been reached for an active job, the relevant report is created, the job is marked inactive and completeed (and optionally NMIS can be controlled once more).

When the scheduled run is over, the job file will not be removed and remain behind; you can therefore reuse a job file by simply updating it with new start and end dates, and target file.

Scheduler  Activation

Scheduler Operation

When the scheduler starts it looks for jobs that are in need of actions (ie. the wanted reporting period has just passed and the relevant report hasn't been created yet) and handles them sequentially. When the scheduled job is finished, the job file is not removed and remains behind (regardless of whether this is a one-off or recurring schedule). After that all known saved reports are checked for pruning: if the report was created with a non-zero keep_for property, and is older than this cutoff then it is removed. This affects only report instances, not the job schedules.

The opReports scheduler The opReports scheduler at this time is not a long-lived daemon process, but  rather meant to be run suitably frequently from cron. The scheduler does log to opreportsopReports.log, but also prints error messages of higher urgency to STDERR, where cron will pick them up and mail them to root (or whatever recipient you have set up). Here is a recommended cron snippet for By default opReports creates an /etc/cron.d/opReports:

Code Block
# Run the opReports Scheduler every 10 minutes
11,21,31,41,51 * * * * root /usr/local/opmantek/bin/opReports-scheduler.pl

If you prefer a per-user crontab, simply remove the "root" column. Naturally, if your scheduled jobs are expected to start or finish on full hours only, then you could reduce the scheduler's run frequency.opreports which triggers the scheduler once every hour.

Remote-Controlling NMIS

This experimental feature is primarily meant for Interface Utilisation Reports at this time and works only for one-off jobs. If your job has control_nmis to true true AND a node_intf_list file, then the following actions are taken:

  • When the job starts, NMIS is reconfigured with global_collect true and with ONLY the wanted nodes and interfaces (identified by the node_intf_list file) set to collect true. All other interface collection activity is disabled with global_nocollect_ifDescr. An nmis.pl type=update run  will then be performed, and the NMIS data collection commences as normal.
  • When the job  completes, NMIS is disabled, with global_collect set to false.

Node and Interface List

The node and interface list can be given in two forms, JSON and Plain Text:

 

...

  • This clearly interferes with recurring schedules and should therefore only be used in isolation.

Optional Settings

The options property can hold sub-properties that fine-tune opReports' behaviour. There are two common options that are shared by all reports:

  • Option title lets you specify a custom title for the report. The value must be a string.
  • Option homelink lets you specify a custom banner, to be displayed only for a standalone HTML report and above the actual report (title and body).
    The value of this option must be a string that contains the raw HTML to be shown.

The WAN report supports option wan_report_level, whose value selects which report detail level should be chosen. This can be given either as the index of the  level or as its name. The page about WAN report detail levels describes this feature in detail.

The Node Health report has option exceptions, which when set to true, changes the report to only display health exceptions and suppress nodes with ok health values.

The Utilisation report recognizes the following four options:

OptionDescription
show_threshold

Default: true. If  this is false, then no over threshold counts  and exceptions are shown. Instead the interface bandwidths, traffic, utilisation and short report period are shown.

show_only_utilDefault: false. Ignored if show_threshold isn't false. If this is set to true, then opReports omits the  bandwidth and traffic columns and shows only the utilisation (and short period).
util_thresholdDefault: 80 (%). Defines the level of utilisation above which it counts as exception.
util_threshold_mincountDefault: 1.  How often the interface utilisation has to be above the threshold value for the interface to be flagged as in exceptional state.

 

Limitations

As of version 3.0 this feature is considered stable but some limitations exist:

  • The NMIS remote control feature is not supported for recurring report schedules, only for one-off reports.
  • The only possible extra action to take at the beginning of a job is enabling NMIS and setting the listed interfaces and nodes for collection.
  • Version 3.0 doesn't support automatic emailing of scheduled reports, only from the command line or from the GUI. This will be added in the next version.
Code Block
{ "testnode": [ "eth0" ], "othernode" : [ 1, 2, "Tunnel20" ] }
  • Plain Text: a text file, one entry per line.
    Each entry must start with the nodename, followed by one or more TAB characters, and one or more interfaces (again separated by TAB characters). If you list a node on multiple lines then all  listed  interfaces will be combined into a single list.
    E.g.

    Code Block
    testnode   2   14    eth0
    othernode  Dialer1
    testnode   17

For both JSON and Plain Text formats, interfaces can be identified by the numeric SNMP interface index, or by the SNMP ifDescr property.

 

...