You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Hi All,


With the coming release of Open-AudIT 3.3.0 we have implemented a new discovery process that scales even better that previously. Even faster discovery times!


With 3.3.0 we have changed to using the discovery queue, not on a per discovery basis as previously, but on a per IP basis. From 3.3.0 onwards, when you click "Execute Discovery", the following happens behind the scenes:

  • The server starts a script that calls /util/queue and instantly returns to the web user (or the API user). It starts the shell script and does not wait for a response before returning.
  • The user then continues on using the web/API as per normal.
  • The shell script calls util/queue - this endpoint only accepts requests from localhost. The resulting function does the following:
    • Check the config for the queue limit. If this has been reached, exit. If it has not been reached, continue.
    • Pop an item from the queue (locking the queue table as it does so). The item is read from the database, then deleted. If no queue items exist, exit.
    • Spawn another script to request util/queue.
    • Execute the item - which on the first time is always "run discovery on subnet".
    • When finished, return to #1.


There are (currently) two types of queue entries. The overall discovery entry, and an entry for each IP to be scanned. The second entry is created by the first. So we run the initial discovery, and for each IP we need to scan (that responds, if that option is chosen), we create another entry to scan that device.


We no longer use the discover_subnet.sh or discover_subnet.vbs scripts at all. We now call Nmap directly from within the Open-AudIT code, which frees us up to have one and only one routine (versus a bash and vbscript). It also makes it easier to code - PHP has much easier to use text parsing than bash and vbscript (in my opinion).


Because of the above, we have created a new configuration item called "discovery_limit" and set it to 20 by default. This means when a discovery is run, it will spawn up to 20 processing instances in parallel. Because of this parallel processing of target IPs, discovery is $discovery_limit times faster. Well, not quite, but you get what I mean. The old way ran a discovery and for each target, sequentially, started a scan. Several scans were run at once, but it was still waiting for an Nmap scan, before handing off to PHP to complete the rest. The new way completes the initial scan and loads all resulting devices into the queue to be processed in parallel. At the end of the day, it's just so much faster.




Windows Users

As well as this, there has been a change targeted specifically to Windows Open-AudIT Servers. Because of the issue's we have run into using the default service account, you will now get a big warning stating you should change the service account to a "real" account. This is because by default the service account cannot access network resources. IE - copy the audit script to thew target and run it. The "old" way of running the script on the Open-AudIT server itself and specifying the target still works and is enabled by a config item - discovery_use_vintage_service, which is set to 'n' by default. One reason for this is that the discovery script contains sections that do now and can not work remotely. Think starting an executable. That won't work as WMI can target the remote machine, but running an executable would run it where the script is running - the Open-AudIT server.

The Default Network Address

Because of our new way of running discovery, we no longer need to set the Default Network Address. The scripts are run on the target devices and create a file (as opposed to submit_online=y). That file is the copied to the server and processed, rather than submitted using the URL (that was created from the default_network_address). The only reason to set the Default Network Address for Discovery is if you're using discovery_use_vintage_service. The now normal use of this is only for the "Audit My PC" functionality.


Change Logs and Non-Current items

And don't forget about our new configuration items for keeping non-current items and creating change logs. More information on these can be found here - Device SubSection Data Retention Options.


I'll update this blog post as we move closer to release.


Mark Unwin.

  • No labels