...
Please note that shadow_file has no effect for commands that have been configured for disk storage.
Large Command Outputs
opConfig can handle Large Command Outputs, but there are some things to take into account:
- Memory used: Files under 16mb did not use much perl memory, but for a 1.5GB file, the perl process could use up to twice that in memory, depending on how it was being handled.
- Storage: MongoDB doesn't support documents greater than 16 megabytes. So command outputs larger than that size will be stored in the path specified in opconfig_external_store configuration option ("<omk_var>/opconfig/external" by default).
- Time: The file has to be transferred over the network, so it will be dependant on the transfer rate.
For example, a file up to 100 MB can be obtained with a cat command, but there are more efficient ways to track the file content. Using a command output will be done with ssh, but opConfig can use scp for that purpose (See below section - Tracking Files). Choosing the right protocol can make a difference.
Tracking Files
opConfig version 3.0.3 introduces a new capability: Arbitrary files can now be downloaded from a node (with SCP), stored and tracked by opConfig.
Here is an snippet from the example command set named file_store.nmis that ships with opConfig:
...