...
Command sets are stored in the single file conf/command_sets.nmis in opConfig before 2.2.4, and since .
Since 2.2.4 opConfig supports this and individual command set files in conf/command_sets.d/ and the files are now .json rather than .nmis.
It is recommended that you use individual files (as that makes things easier to maintain and more robust, e.g. should one file have syntactical problems). For backwards compatibility the old single file is consulted first, then any individual ones. opConfig loads a command set when it's seen for the first time, and any subsequent occurrences of a clashing config set (i.e. same name) will be ignored but logged.
...
Here is an excerpt from the default command set that that ships with opConfig :(in .nmis, pre 2.2.4 format).
| Code Block | ||
|---|---|---|
| ||
'IOS_HOURLY' => {
'os_info' => {
'version' => '/12.2|12.4|15.\d+/',
'os' => 'IOS'
},
'purging_policy' => {
'keep_last' => 1000,
'purge_older_than' => 2592000, # 30 days
'autoprotect_first_revision' => 'true',
},
'scheduling_info' => {
'run_commands_on_separate_connection' => 'false',
},
'commands' => [
{
'multipage' => 'true',
'privileged' => 'false',
'command' => 'show interfaces',
'tags' => [ 'HOURLY', 'performance', 'interfaces' ],
},
{
'multipage' => 'true',
'privileged' => 'false',
'command' => 'show ip interface',
'tags' => [ 'HOURLY', 'detect-change', 'configuration' ],
},
... and lots more |
Defining which nodes a command set applies to
This is normally done by filtering by OS info, but not necssarily limited to OS info only:Below is much the same, but in the post 2.2.4, json format.
| Code Block | ||
|---|---|---|
| ||
#filter using both regular expression and string equality checks: 'os_info' => { 'version' => '/12.2|12.4|15.0/', 'os' => 'IOS' }, # or just the specific os family/type and that's all: 'os_info' => { 'os' => '/(Linux|CentOS|Ubuntu)/' }, |
The examples above specify that the node in question must have an os_info property set, with sub-properties os and version in the first example and just os in the second.
Plain string values indicate string comparison, but regular expressions can be given too. A node is only considered for this particular command set if all filtering expressions match. You can filter on any node property, not just properties from os_info (but the default command sets only use os_info).
Prior to version 3.1.1, opConfig considered command sets without filter blocks as disabled; for these versions you may want a 'wildcard' filter matching anything whatsoever, which can be achieved by adding an os_info filter block with a 'match-everything' regular expression for e.g os,'os' => '/.*/'.
From opConfig 3.1.1 onwards a command set without filter is interpreted as to apply to all nodes without any restriction.
...
"IOS_HOURLY" : {
"scheduling_info" : {
"run_commands_on_separate_connection" : "false",
"attempt_timeout_recovery" : 1
},
"purging_policy" : {
"purge_older_than" : 2592000,
"keep_last" : 1000,
"autoprotect_first_revision" : "true"
},
"commands" : [
{
"command" : "show interfaces",
"privileged" : "false",
"tags" : [
"HOURLY",
"performance",
"interfaces"
],
"multipage" : "true"
}, |
Defining which nodes a command set applies to
This is normally done by filtering by OS info, but not necessarily limited to OS info only:
| Code Block | ||
|---|---|---|
| ||
# Filter using both regular expression and string equality checks:
'os_info' => {
'version' => '/12.2|12.4|15.0/',
'os' => 'IOS'
},
# Or just the specific os family/type and that's all:
'os_info' => {
'os' => '/(Linux|CentOS|Ubuntu)/'
}, |
The examples above specify that the node in question must have an os_info property set, with sub-properties os and version in the first example and just os in the second.
Plain string values indicate string comparison, but regular expressions can be given too. A node is only considered for this particular command set if all filtering expressions match. You can filter on any node property, not just properties from os_info (but the default command sets only use os_info).
Prior to version 3.1.1, opConfig considered command sets without filter blocks as disabled; for these versions you may want a 'wildcard' filter matching anything whatsoever, which can be achieved by adding an os_info filter block with a 'match-everything' regular expression for e.g os,'os' => '/.*/'.
From opConfig 3.1.1 onwards a command set without filter is interpreted as to apply to all nodes without any restriction.
Controlling how the command output is stored
| Anchor | ||||
|---|---|---|---|---|
|
By default opConfig stores all command outputs in its MongoDB database, and only there.
In recent versions two new options let you fine-tune the behaviour and make use of on-disk storage where desired:
- Shadowing outputs on disk is controlled by the option
shadow_file, - and the choice between database storage and disk storage is made via the option
store_internal.
Disk or Database?
If you set the store_internal option to "true" or omit it altogether, then all command outputs are stored in the database.
With this option set to false, opConfig creates a separate file on disk for each revision of the command.
The storage location for such files is configured with the config option opconfig_external_store (default /usr/local/omk/var/opconfig/external).
The actual files are then stored in the directory <opconfig_external_store>/<node name>/<command name>/<revision>.
In version 3.1.1 and newer, an additional symbolic link latest that points to the most recent revision is maintained in that directory.
For externally stored data the visible 'command output' in the GUI is made up from the size and the SHA256 checksum of the contents, and change detection (and the GUI) uses this data instead of the (possibly binary or huge) file contents. This produces much more coarse change detection, but works with binary files. In the GUI you'll see the made up 'command output', and a button to download the actual contents.
All other opConfig capabilities work normally for externally stored data; e.g. scheduling, tags, purging of old revisions, revision creation itself and so on.
Please note that in versions before 3.1.1 store_internal is only supported for Tracking Files.
Commands or Files that cannot fit into the Database?
MongoDB doesn't support documents greater than 16 megabytes. In opConfig 3.1.1 and newer, any command output that is larger than this limit is automatically reconfigured for storage on disk, i.e. store_internal is overridden and set to "false" for just this one revision.
Both Disk and Database?
In opConfig 3.1.1 an option for shadowing command output on disk was added: if you set the property shadow_file to true (in the command's block, or in the command set's scheduling_policy section), then opConfig will store the data both in the database and also on disk (in the same place as externally stored command outputs).
Please note that shadow_file has no effect for commands that have been configured for disk storage.
Large Command Outputs
opConfig can handle Large Command Outputs, but there are some things to take into account:
...
By default opConfig stores all command outputs in its MongoDB database, and only there.
In recent versions two new options let you fine-tune the behaviour and make use of on-disk storage where desired:
- Shadowing outputs on disk is controlled by the option
shadow_file, - and the choice between database storage and disk storage is made via the option
store_internal.
Disk or Database?
If you set the store_internal option to "true" or omit it altogether, then all command outputs are stored in the database.
With this option set to false, opConfig creates a separate file on disk for each revision of the command.
The storage location for such files is configured with the config option opconfig_external_store (default /usr/local/omk/var/opconfig/external).
The actual files are then stored in the directory <opconfig_external_store>/<node name>/<command name>/<revision>.
In version 3.1.1 and newer, an additional symbolic link latest that points to the most recent revision is maintained in that directory.
For externally stored data the visible 'command output' in the GUI is made up from the size and the SHA256 checksum of the contents, and change detection (and the GUI) uses this data instead of the (possibly binary or huge) file contents. This produces much more coarse change detection, but works with binary files. In the GUI you'll see the made up 'command output', and a button to download the actua contents.
All other opConfig capabilities work normally for externally stored data; e.g. scheduling, tags, purging of old revisions, revision creation itself and so on.
Please note that in versions before 3.1.1 store_internal is only supported for Tracking Files.
Commands or Files that cannot fit into the Database?
- Memory used: Files under 16mb did not use much perl memory, but for a 1.5GB file, the perl process could use up to twice that in memory, depending on how it was being handled.
- Storage:
- MongoDB doesn't support documents greater than 16 megabytes.
...
- So command outputs larger than that size will be stored in the path specified in opconfig_external_store configuration option ("<omk_var>/opconfig/external" by default).
- Time: The file has to be transferred over the network, so it will be dependant on the transfer rate.
For example, a file up to 100 MB can be obtained with a cat command, but there are more efficient ways to track the file content. Using a command output will be done with ssh, but opConfig can use scp for that purpose (See below section - Tracking Files). Choosing the right protocol can make a difference.
Both Disk and Database?
In opConfig 3.1.1 an option for shadowing command output on disk was added: if you set the property shadow_file to true (in the command's block, or in the command set's scheduling_policy section), then opConfig will store the data both in the database and also on disk (in the same place as externally stored command outputs).
Please note that shadow_file has no effect for commands that have been configured for disk storage.
Tracking Files
opConfig version 3.0.3 introduces a new capability: Arbitrary files can now be downloaded from a node (with SCP), stored and tracked by opConfig.
Here is an snippet from the example command set named file_store.nmis that ships with opConfig:
...
opConfig 2.2 and newer can raise an event with NMIS if a change was detected for a node and a particular command. To . The event name is "Node Configuration Change Detected" and includes details of the change which has been detected. To activate this feature you have to give the command in question both the tags detect-change and report-change. You may also give a fixed event severity (values as defined in NMIS), like in this example:
...
eg. act=diff_command_outputs node=node1 command="show run" revision_1=10 revision_2=13
import_audit
This action imports audit information from Open-AudIT Enterprise, for all devices known to Open-AudIT.
This requires a licensed Version 1.4 of Open-AudIT Enterprise or newer. The audit information will show up in opConfig as command "audit", and the command output will be the full audit data set in JSON format. Nodes will be consolidated as much as possible with your currently existing node definitions.
To enable this functionality, set the necessary options in conf/opCommon.nmis and run opconfig-cli.pl act=import_audit periodically or as a one-off. The required config settings are as follows:
| Code Block |
|---|
'opconfig_audit_import' => 1, # to enable this altogether 'opconfig_audit_import_url_base' => "http://my.oae.system.com/omk/oae/", 'opconfig_audit_import_user' => "nmis", # needs ro-access 'opconfig_audit_import_password' => "somepassword", |
...