Configuration Options

Credential Sets

Credentials for all connections made by opConfig are stored encrypted in the database, and are configurable from the opConfig GUI: Menu System, Edit Credential Sets. Each credential sets has a name, and can be shared by any number of nodes.

Connections and OS Info

Connections define which devices/nodes opConfig should connect to, and which commands to run on that node. opConfig will connect to any node that has valid settings for personality, transport and credential set.

• "Personality" identifies which set of prompts, strings, line-ending conventions etc. a node is subject to; for example, the 'ios' personality works only on Cisco IOS devices, while the 'bash' personality covers just about all Unix systems with the bash shell.
• "Transport" is one of Telnet or SSH. At this point in time, only password-based authentication is supported for SSH.
• "Credential Set" is the name of an existing credential  set that defines user name and password(s) for authenticating to a node.

To configure these settings go to the System Menu, Edit Nodes, select a node for edit/delete, then go to the Connections section. You'll see the relevant options shown with explanations in tooltips.

In addition to these fixed characteristics of a node, opConfig also makes use of the "OS Info" section: the entries in this section roughly define the node's Operating System, Platform and related characteristics. Generally all of opConfig's command sets are specific to OS or platform.

If you Import a Node from NMIS (or Refresh a Node from NMIS), then the OS information for this node is also imported from NMIS. You can, however, override those settings in the GUI.

Command Sets

Each command set defines two types of information:

• information for filtering out which devices/nodes the commands will be run on,
• and the actual commands to be run, and whether the commands should be run sequentiall in a single session/connection or be run separately on their own.

Here is an example command set, I will break it down in the sections that follow:

'IOS_DAILY' => {
 'os_info' => {
   'version' => '/12.2|12.4|15.0/',
   'os' => 'IOS'
 },
 'aging_info' => {
   'age' => 'forever'
 },
 'scheduling_info' => {
   'run_commands_on_separate_connection' => 'false'
 },
 commands => [
 {
   'tags' => [ 'config','version','troubleshooting', 'detect-change' ],
   'command' => 'show version',
   'privileged' => 'false',
   'multipage' => 'true',
   'run_command_on_separate_connection' => 'false',
   'command_filters' => [
     '/uptime is/'
   ]
 }
}} 

Defining which nodes a command set applies to:

Filtering by OS info

#filter using a regular expression:
'os_info' => {
 'version' => '/12.2|12.4|15.0/',
 'os' => 'IOS'
},
# or just the specific os and that's all:
'os_info' => { 
 'os' => 'Linux'
},  

A regular expression can be used in any filtering field.  Any hash or field can be defined in a connection and then filtered on in the command set (not just the ones that have been shown).

Making opConfig detect changes in a command output:

Not all output is worthy of change detection, configuration information generally is while performance information generally is not.  The configuration for a router, for example, is - but the usage counters on an interface is not.  Commands that have the tag detect-change will have change detection run on them by opConfig.  If the output is different from the previously stored output, a new revision is stored along with what the changes were.  If the tag is not defined, a new revision is stored (if the output is different) but no changes are tracked (and will not be reported in the GUI as a configuration change).

'tags' => ['detect-change'],

There is one more issue to resolve.  Not all changes found may be considered important or relavent.  To ignore unimportant changes a set of command filters can be added to a command:

'command_filters' => [
 '/uptime is/'
]

If the regular expression matches a change that change is considered unimportant and will not be tracked by the change management system (and so the GUI will not report it as a change)

Grouping commands into sessions:

By default all commands for a device will be run one after another within the same session/connection to that device.  This will generally save time as connecting to the device often takes longer than running all the commands on it.  This is not always the case and there will be certain commands that run for a longer period of time that should have their own connection, for example statistics gathering with iostat or vmstat.

If you would like to change the default behaviour and have opConfig run all commands in all command set in separate session/connection, modify opCommon.nmis like this:

# to run each command in it's own session, change this to true
 'opconfig_run_commands_on_separate_connection' => 'false',

It makes a lot more sense to tell opConfig to run a set of commands on their own or, in most cases, just a single command on it's own.  Running a command or set of commands on their own session/connection:

# to run all commands in one command set define this key to be true - note the S in run_commandS_...
'scheduling_info' => {
 'run_commands_on_separate_connection' => 'true'
},

# to run just a specific command on its own - no S in run_command_...
commands => [
 {
 'command' => 'show version',
 'run_command_on_separate_connection' => 'true',
 }]

opConfig Concepts

The main concepts to bear in mind are nodes, commands, changes and revisions.

1. Nodes are computer systems that opConfig knows about, and which it is configured to run commands for.
2. As opConfig needs to connect to the node in question and execute commands on that node, the node needs to be configured with access credentials. In opConfig these are stored independent from the nodes in what opConfig calls credential sets.
3. Commands are normally command line constructs which will be executed on the node in question. The command output is captured and stored by opConfig.
   There are some 'passive' commands (like audit-import) which are not run on a node, but which are associated with a node.
4. A command can be one-shot command which is not analyzed in great detail (e.g. a process listing or some other diagnostic command), and whenever such a one-shot command is executed, its output is roughly compared against the previous revision, and if different it's saved as a new revision in opConfig.
5. A command can be marked for change detection (e.g. a listing of installed software), in which case opConfig creates detailed records of what the changes are if there are differences between the current command output and the most recent revision for this command.
6. Commands can be grouped and collected into what opConfig calls a "command set". Command sets can be configured to apply only to particular platforms or OS versions.

How to add (or modify) a Node

To tell opConfig to run commands for a node it needs to be told about the node's existence and what properties the node has (e.g. what platform, what OS, whether to use Telnet or SSH to contact the node and so on). Adding a node for opConfig can be done using the GUI or the command line tools opconfig-cli.pl and opnode_admin.pl. You can provide the node's information manually, or you can import a node's info from NMIS.

opConfig will connect to any node (and run commands for it) that has valid connection settings and which isn't explicitely disabled for opConfig.

Using the GUI

In the GUI simply select the System menu, then Edit Nodes. The buttons "Import new Nodes from NMIS" and "Add Node"  let you create node records, which will however need some adjustment for full functionality. Most entries on these editing pages have tooltips with explanations.

If you edit an imported, existing node or click on Add Node you'll likely see some entries in the red "Configuration Problems" tab. Here is an example:

Image Added

The problem reports are fairly self-explanatory (and clickable), but let's go over them quickly:

• To connect to a node, opConfig needs to know what Transport to use (Telnet or SSH). This is configured in the "Connection" tab.
  At this point in time, opConfig supports only Telnet and SSH, and for SSH only password-based authentication is supported.
• The Personality is also required to tell opConfig what kind of commands this node understands, e.g. whether it's a Unix-like system with a real shell or whether it's a Cisco IOS device and so on. The Personality includes information about the prompts, line-ending conventions etc. a node is subject to; for example, the 'ios' personality works only on Cisco IOS devices, while the 'bash' personality covers just about all Unix systems with the bash shell.
• Because the default command sets that opConfig ships with do use the Operating System name to select which command sets to apply, it's vital that you tell opConfig about your new node's OS.
  If a node is imported from NMIS (or Open-AudIT Enterprise) then the OS Info is prefilled as much as possible (but can be modified by you, of course). If there is no or incorrect OS information, then opConfig will not run any or the right command sets on your node.
• And, last but not least: interactive connections to nodes clearly require authentication and authorization in the form of access credentials, hence you must tell opConfig which Credential Set should apply to your new node.

Import (and discovery) from the Command Line

opconfig-cli.pl can import nodes from NMIS, and offers a credential set and transport discovery option. Simply run opconfig-cli.pl without options to see a brief usage help.

To import you'd run opconfig-cli.pl act=import_from_nmis (optionally limited to the names of known nodes with an argument of nodes=nodeX,nodeY). If you have already setup credential sets, then you can let opConfig guess which to use for your node using opconfig-cli.pl act=discover node=TheNewNodeName - if none of the Transport+Credential Set combinations work for the node, opconfig-cli.pl will print an error message.

General Node management from the Command Line

The tool opnode_admin.pl lets you list, export, create, modify or delete nodes from the command line. Editing operations involve an export to a JSON file, which you then modify using an editor or other tool of choice, followed by an update of the node using that JSON file. The opnode_admin.pl tool is pretty self-explanatory, and behaves almost identically to the NMIS node_admin.pl tool which is documented extensively here.

Managing Credential Sets

Credentials for all connections made by opConfig are configurable from the opConfig GUI (only): Select the menu System, then Edit Credential Sets. Credential sets can be shared by any number of nodes.

Each credential set has to have a unique name, by which it is referenced in the nodes' connection settings. The description field is self-explanatory and optional.

A credential set has to specify a User Name property, which is used when logging in to the nodes the set applies to. At this time, opConfig supports only password-based authentication at the node, and the Password property of the credential  set establishes the primary password for this user name. Future versions will support SSH Key-based authentication, as well as other mechanisms.

Some commands cannot be performed by an unprivileged user, which is why opConfig also supports elevating the privileges on demand. To control this, a credential set can optionally include a Superuser/Privileged/Enable Password. Depending on the node's platform and personality, different mechanisms will be used to gain increased privileges:

• On Cisco IOS devices, this password  is used with the enable command.
• With personality bash (the default for Unix-like systems), the command sudo is used to become the superuser. Sudo therefore needs to be installed and configured on such nodes, and the User Name in question needs to be authorized for sudo.

Naturally not all commands require elevated privileges; see the section on Command Sets for how to determine and configure those.

Please note that the Credential  Set editing dialogs do never show existing passwords (or their legth or existence); You can only overwrite password entries. All credential sets are stored in the database in encrypted form.

Managing Command Sets

As mentioned above opConfig lets you organize whatever commands you'd like it to run into an arbitrary number of groups which we call command sets.

A command set definition consists of these types of information:

• meta-information like the command set's name,
• optional control information like scheduling constraints and an expiration policy for the commands in that set,
• filters for selecting which devices/nodes the command set should apply to,
• and finally the list of the actual commands to be run.

Here is an excerpt from the default command set that  ships with opConfig:

  'IOS_HOURLY' => {
    'os_info' => {
      'version' => '/12.2|12.4|15.\d+/',
      'os' => 'IOS'
    },
        'purging_policy' => {
            'keep_last' => 1000,
            'purge_older_than' => 2592000, # 30 days
            'autoprotect_first_revision' => 'true', 
        },    
        
        'scheduling_info' => {
      'run_commands_on_separate_connection' => 'false',
    },    
    'commands' => [
      {
        'multipage' => 'true',
        'privileged' => 'false',
        'command' => 'show interfaces',
        'tags' => [ 'HOURLY', 'performance', 'interfaces' ],
      },
      {
        'multipage' => 'true',
        'privileged' => 'false',
        'command' => 'show ip interface',
        'tags' => [ 'HOURLY', 'detect-change', 'configuration' ],
      },
... and lots more

Defining which nodes a command set applies to

This is normally done by filtering by OS info:

#filter using both regular expression and string equality checks:
'os_info' => {
 'version' => '/12.2|12.4|15.0/',
 'os' => 'IOS'
},
# or just the specific os family/type and that's all:
'os_info' => { 
 'os' => '/(Linux|CentOS|Ubuntu)/'
},

The examples above specify that the node in question must have an os_info property set, with sub-properties  os and version in the first example and just os in the second.

Plain string values indicate string comparison, but regular expressions can be given too. A node is only considered for this particular command set if all filtering expressions match. You can filter on any node property, not just properties from os_info (but the default command sets only use os_info).

Grouping commands into sessions

By default all commands for a node will be run sequentially, one after another within the same session/connection to that device.  This can save time as connecting to the device often takes longer than running all the commands one by one.  But this is not always the case,  and there are certain commands that run for a longer period of time (for example system statistics gatherers like vmstat or top). These commands would cause long delays in a sequential run, therefore opConfig lets you specify which commands should have their own separate connections. Separate connections can be opened in parallel.

If you would like to change the default behaviour and have opConfig run all commands in all command set in separate session/connection, simply modify opCommon.nmis like this:

# to run each command in it's own session, change this to true
 'opconfig_run_commands_on_separate_connection' => 'false',

It makes a lot more sense to tell opConfig to run a set of commands on their own or, in most cases, just a single command on its own.  Here is how to make a command or command set have their own sessions/connections:

# for all commands in a set, define this in the command set - note the S in run_commandS_...!
'scheduling_info' => {
 'run_commands_on_separate_connection' => 'true'
},

# for just a specific command, set this for the command in question - no S in run_command_...!
commands => [
 {
 'command' => 'show version',
 'run_command_on_separate_connection' => 'true',
 }]

How long should captured command output be kept

opConfig 2.2. (and newer) have a flexible purging subsystem which is described in detail on a separate page here. The example above shows roughly how it's controlled: a command or command set can have a  section called purging_policy which controls whether and when a revision should be removed from the database.

What constitutes a change, and when should opConfig create new revisions

Not all output is worthy of change detection; configuration information generally is while performance information generally is not. The configuration for a router, for example, is - but the usage counters on an interface likely are not. As mentioned above, opConfig can deal with both of these types of commands, the "one-shot" ones as well as the change-indicating ones.

In a command set you tell opConfig which commands belong to what class by setting (or omitting) the tag detect-change.

If the tag is present, then change detection is run on this command's output: If and only if  the output is different from the previously stored revision, a new revision is stored along with detailed information of what the changes were. If the tag is not defined, then a new revision is created if the output is different - but no detailed changes are tracked, and the opConfig GUI will only report the command's revision in the list of "commands" but not in the list of "configuration changes".

'tags' => ['detect-change'],

Related to this is the question of what changes that were found should be considered important or relevant.  opConfig can "ignore" unimportant differences in a command's output if you provide it with a set of command filters for a command:

'privileged' => 'false',
'command' => 'ifconfig -a',
'tags' => [ 'DAILY', 'configuration', 'detect-change' ],
  'command_filters' => [
    '/RX packets/',
    '/TX packets/',
    '/RX bytes/',
    '/TX bytes/'
  ]

In the example above, the output of the "ifconfig -a" command would be checked and any changed lines that match TX/RX packets or TX/RX bytes (i.e. the interface counters) are ignored. Only other changes that are left after applying the filters are checked to figure out whether to create a new revision or ont.

Please note that command filters are possible for both one-shot commands and commands with change-detect enabled,  and behave the same for both.

How to categorize command sets (and why)

As seen in the examples above, commands can have a number of pre-defined tags like detect-change or report-change; but you are encouraged to set your own custom tags to characterize the commands according to your preferences.

The default command set shipped with opConfig uses two primary tags for categorization, DAILY and HOURLY.These are used in the example cron invocations of "opconfig-cli.pl act=run_command_sets" so that all commands with tag DAILY are run only once a day, whereas the HOURLY ones run once an hour. Commands with neither tag will not run periodically, for example there are numerous commands tagged with troubleshooting which will only be run in reaction to an event detected in opEvents.

Using tags in this fashion for categorization is much more convenient than having to tell opconfig-cli.pl which command sets to run by name (which is also possible).

Raising Events when changes are detected

opConfig 2.2 and newer can raise an event with NMIS if a change was detected for a node and a particular command. To activate this feature you have to  give the command in question both the tags detect-change and report-change. You may also give a fixed event severity (values as defined in NMIS), like in this example:

'privileged' => 'false',
'command' => 'chkconfig',
'tags' => [ 'DAILY', 'configuration', 'detect-change', 'report-change' ],
'report_level' => 'Minor',

In this case, the Redhat/Centos command chkconfig (= list of system services to automatically start on boot) will be checked for changes, and if any are found then a "Node Configuration Change" event with the context node in question, the element "chkconfig" and the serverity "Minor" will be raised in the local NMIS.

If you want a more dynamic event severity, then you can use report_level_min_changes which selects a severity based on the number of changes that were found:

{        
 'privileged' => 'true',
 'command' => 'vgdisplay',
 'tags' => [ 'DAILY', 'configuration', 'detect-change', 'report-change' ],
 'report_level_min_changes' => {
   1 => "Normal",
   3 => "Minor",
   10 => "Major" },
}

In this example, changes in the vgdisplay command output would result in an event of severity Normal if there are 1 or 2 changes, Minor for 3 to 9 changes, and Major for 10 or more.

General Usage of opconfig-cli.pl

...

opconfig-cli

Information on what commands are supported is printed when no options are specified.  All options support debug=true for debug output and debug=9 for extremely verbose output.  

...