DRAFT
Summary
In this guide we will migrate the data from NMIS8 and OMK Applications to nmis9 and OMK Applications.
- Server: uburnto (ubuntu 18) → deb-n-burn (debian 10)
- nmis 8.7.2 → nmis 9.2.2
- opCharts 2.5.1 → opCharts 3
- opConfig 3.5.2 → opConfig 4
- opEvents 2.6.3 → opEvents 3
- opHA 2.2.2 → opHA 3
- opReports 3.4.2 → opReports 4
Options
- Use a VM for the nmis9 server: We would need to run additional steps to migrate the data.
- Install the apps one the data is migrated: The installers will upgrade the data to the new format.
Upgrade process: Using nmis9 VM
Prerequisites
- NMIS 9 and OMK apps are installed on the server.
Step 1. Stop all daemons
We first need to make sure all the daemons are stopped in the destination server:
sudo service nmis9d stop sudo service omkd stop sudo service opeventsd stop sudo service opchartsd stop sudo service opconfigd stop
Step 2. Move and adapt NMIS files
Configuration Files
Copy
Copy the configuration directory from the source server.
Create a conf backup first:
mv /usr/local/nmis9/conf /tmp/nmis9/conf
This can be done really easy with rsync:
rsync -r root@uburnto.opmantek.net:/usr/local/nmis8/conf /usr/local/nmis9
Configure
We need to make some adjustments to the file. This is:
- Replace all nmis8 occurrences by nmis9.
- Remove sql section.
- Add database section.
'database' => {
'db_name' => 'nmisng',
'db_password' => 'op42flow42',
'db_port' => '27017',
'db_query_timeout' => 5000,
'db_server' => 'localhost',
'db_username' => 'opUserRW',
'db_never_remove_indices' => [ 'nodes' ],
},
...
'<nmis_base>' => '/usr/local/nmis9',
'<url_base>' => '/nmis9',
'<cgi_url_base>' => '/cgi-nmis9',
- Review important parameters, as the number of workers (workers parameter in config). We can use this rule to get a starter number, but this will depend on the server resources, the amount of data collected by the node, etc. If the server is too slow we would need to decrease the number. If there are nodes with more than 3x late collects (Check the polling summary), the number should be increased:
(Number of nodes x AVG Collect time) / 300
300 _ Default polling time.
- Then, fix files permissions:
/usr/local/nmis9/bin/nmis-cli act=fixperms
Check
Even with the daemon stop, we can check if the configuration is working. We can access the GUI:
http://server/cgi-nmis9/nmiscgi.pl
If we see any 500 Error, we can review the apache logs, usually in the path /var/log/apache2/error.log
Model Files
We can move all the model customisations in the models-custom directory. We can use rsync for this again.
NOTE Some models may need to be adjusted. Specially if they are using nmis internal functions.
Plugins
We have moved the conf/plugins directory using rsync. But the custom plugins need to be adapted to NMIS9.
Here you can find further information.
Step 3. Move and adapt OMK files
Configuration Files
Copy
Copy the configuration directory from the source server.
Create a conf backup first:
mv /usr/local/omk/conf /tmp/omk/conf
This can be done really easy with rsync:
rsync -r root@uburnto.opmantek.net:/usr/local/omk/conf /usr/local/omk
Convert to JSON
It can be done with the following command:
/usr/local/omk# /usr/local/omk/bin/opcommon-cli.exe act=convert_json_dir dir=conf
Configure
TODO
Step 4. Copy RRDs
This can be done really easy with rsync:
rsync -r root@uburnto.opmantek.net:/usr/local/nmis8/database/nodes /usr/local/nmis9/database
Then fix permissions:
/usr/local/nmis9/bin/nmis-cli act=fixperms
Run script to adapt names
This script will adapt nmis8 rrd names to nmis9 (From nmis 9.2.1):
/usr/local/nmis9/admin/node_admin.pl act=move-nmis8-rrd-files {node=nodeX|ALL|uuid=nodeUUID} [remove_old=1] [force=1]
Step 5. Import Nodes
We have copied the nodes file from nmis8 using rsync. We would need to import the nodes now:
/usr/local/nmis9/admin/node_admin.pl act=import_bulk nodes=/usr/local/nmis9/conf/Nodes.nmis debug=true
Step 6. Move OMK Database
- Stop services in origin server
- Generate mongodump:
mkdir /data/mongodump mongodump -u=opUserRW -p=op42flow42 --out=/data/mongodump
- copy the data with rsync:
rsync -r root@uburnto.opmantek.net:/data/mongodump /data
- Restore:
mongorestore -u opUserRW -p op42flow42 mongodump/
Step 7. OMK Applications considerations
opEvents
...
opConfig
...
opReports
NMIS8 opReports and NMIS9 opReports work the same way:
Report Schedules are stored in /usr/local/conf/schedule
Reports caching is stored in /usr/local/omk/var/opreports
Reports are stored in /data/omk/var/reports (unless otherwise specified by the user)
If you followed the instructions above, your Report Schedules are back in place, as you've copied over the entire omk-old/conf directory in step five. The /data/omk/var reports directory should still be where it was at the beginning of the upgrade, as it was never changed.
We recommend coping the ./opreports/known_reports_cache directory and the ./opreports/selftest.json file into the new application.
cp -rf /usr/local/omk-old/var/opreports/known_reports_cache /usr/local/omk/var/opreports cp /usr/local/omk-old/var/opreports/selftest.json /usr/local/omk/var/opreports
opHA
...
Step 8. Start daemons
...
Step by Step Manual Upgrade Process
In short
Complete Install of NMIS9 HERE
Move aside the omk directory and install the new NMIS9 compatible application. Once the installation has finished rename the new omk/conf folder and copy over the old omk/conf folder. Convert the .nmis files in omk/conf/ directory into .json using the opcommon-cli tool. Edit the new omk/conf/opCommon.json file replacing any "nmis_dir" and "nmis8" entries for "nmis9_dir" and "nmis9". Lastly ensure the "load_applications" entry in this file shows only the new NMIS9 application previously installed and restart the omk daemon. We recommend starting this upgrade process with either opCharts or opConfig.
In detail
Install NMIS9, should install without any issues
Rename the omk directory
mv /usr/local/omk /usr/local/omk-old
Run the installer for the new NMIS9 application, e.g. opCharts
Rename the new omk/conf directory
mv /usr/local/omk/conf /usr/local/omk/conf-original
Copy the omk-old/conf directory into the new omk/
cp -rf /usr/local/omk-old/conf /usr/local/omk
Convert the .nmis files in omk/conf/ directory into .json using the opcommon-cli tool
/usr/local/omk/bin/opcommon-cli.exe act=convert_json_dir dir="/usr/local/omk/conf/"
Change any "nmis_dir" and "nmis8" entries to "nmis9_dir" and "nmis9" in the omk/conf/opCommon.json file. You can do this manually or by using the following Perl regex
perl -p -i -e 's/nmis_dir/nmis9_dir/g' /usr/local/omk/conf/opCommon.json perl -p -i -e 's/nmis8/nmis9/g' /usr/local/omk/conf/opCommon.json perl -p -i -e 's/nmis_logs/nmis9_logs/g' /usr/local/omk/conf/opCommon.json
Change the "load_applications" entry in the opCommon.json file to only show the NMIS9 application recently installed. You can do this manually or by running the required patch_config commands to delete each application from the "load_applications" entry, except for the one that was recently installed.
# E.g.: Previously installed applications: opConfig, opCharts, opEvents, opHA, opReports and Open-AudIT # Recently installed application: opConfig /usr/local/omk/bin/patch_config.pl -b /usr/local/omk/conf/opCommon.json /omkd/load_applications-=opCharts # -b = Creates backup (opCommon.json.prepatch) /usr/local/omk/bin/patch_config.pl /usr/local/omk/conf/opCommon.json /omkd/load_applications-=opEvents /usr/local/omk/bin/patch_config.pl /usr/local/omk/conf/opCommon.json /omkd/load_applications-=opHA /usr/local/omk/bin/patch_config.pl /usr/local/omk/conf/opCommon.json /omkd/load_applications-=opReports /usr/local/omk/bin/patch_config.pl /usr/local/omk/conf/opCommon.json /omkd/load_applications-=Open-AudIT # If the recently installed application (eg.: opConfig) is deleted by mistake, you can add it back by running the follow patch_config command /usr/local/omk/bin/patch_config.pl /usr/local/omk/conf/opCommon.json /omkd/load_applications+=opConfig
Restart omkd
systemctl restart omkd.service
Install the rest of the NMIS9 required applications
About opReports
NMIS8 opReports and NMIS9 opReports work the same way:
Report Schedules are stored in /usr/local/conf/schedule
Reports caching is stored in /usr/local/omk/var/opreports
Reports are stored in /data/omk/var/reports (unless otherwise specified by the user)
If you followed the instructions above, your Report Schedules are back in place, as you've copied over the entire omk-old/conf directory in step five. The /data/omk/var reports directory should still be where it was at the beginning of the upgrade, as it was never changed.
We recommend coping the ./opreports/known_reports_cache directory and the ./opreports/selftest.json file into the new application.
cp -rf /usr/local/omk-old/var/opreports/known_reports_cache /usr/local/omk/var/opreports cp /usr/local/omk-old/var/opreports/selftest.json /usr/local/omk/var/opreports