I confirmed that data= is not at the beginning of the xml files.
Of note, I did try manually pushing one to the server using curl from the command line directly and got a response starting with "Disallowed Key Characters in" - this suggested some xml validation issues. I commented out the regex match in MY_Input as a temporary troubleshooting step to get by this.
I can provide one of the xml files privately if there's a means here, but I can't share it publicly obviously. If this is a matter of a linux process being identified that's producing problem text, I can't find it. The XML files pass XML validation when I've tested them.
I also don't want to get bogged down on the submission process. These fail even if I take the contents of an XML file and submit it manually via text inside the open-audit app itself.
Output from the requested command - ( bracketed text replaces actual server names, etc below...):
[root@[SERVER] tmp]# ./audit_linux.sh submit_online=y create_file=n url=http://[Actual.URL.of.Server]/open-audit/index.php/input/devices debugging=5
Starting audit
----------------------------
Open-AudIT Linux audit script
Version: 3.3.2
----------------------------
My PID is 6105
Audit Start Time 2020-07-24 14:00:28
Create File n
Submit Online y
Debugging Level 5
Discovery ID
Org Id
Script Name audit_linux.sh
URL http://[Actual.URL.of.Server]/open-audit/index.php/input/devices
File /var/tmp/[SERVER]-20200724140028.xml
----------------------------
System Info
Policy Info
BIOS Info
Processor Info
Memory Info
Motherboard Info
Optical Drives Info
Video Cards Info
Sound Cards Info
Shares Info
Network Cards Info
Hard Disk Info
Guest (Docker, Proxmox, LXC) Info
Docker Info
Log Info
Environment Variable Info
Swap Info
User Info
Group Info
Software Info
Service Info
systemd services
init.d services
Server Info
find: [REMOVED] : No such file or directory
Route Info
Netstat Info
Custom Files Info
Audit Generated in '48' seconds.
Submitting results to server using cURL
URL: http://[Actual.URL.of.Server]/open-audit/index.php/input/devices
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 294k 100 252 0 0 35328 0 --:--:-- --:--:-- --:--:-- 50400
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="https://[Actual.URL.of.Server]open-audit/index.php/input/devices">here</a>.</p>
</body></html>
Audit Completed in '49' seconds.
Your server is responding to https, but you're using http in your command line arguments. Change that.
That explains the 302. That was because I copy-pasted the command you requested that I run. Everywhere else, we use https. I just didn't notice that your request had http in it. Now I get the 503 that I originally reported.
[root@[SERVER] tmp]# ./audit_linux.sh submit_online=y create_file=n url=https://[Actual.URL.of.Server]/open-audit/index.php/input/devices debugging=5
Starting audit
----------------------------
Open-AudIT Linux audit script
Version: 3.3.2
----------------------------
My PID is 18530
Audit Start Time 2020-07-27 08:44:32
Create File n
Submit Online y
Debugging Level 5
Discovery ID
Org Id
Script Name audit_linux.sh
URL https://[Actual.URL.of.Server]/open-audit/index.php/input/devices
File /var/tmp/[SERVER]-20200727084432.xml
----------------------------
System Info
Policy Info
BIOS Info
Processor Info
Memory Info
Motherboard Info
Optical Drives Info
Video Cards Info
Sound Cards Info
Shares Info
Network Cards Info
Hard Disk Info
Guest (Docker, Proxmox, LXC) Info
Docker Info
Log Info
Environment Variable Info
Swap Info
User Info
Group Info
Software Info
Service Info
systemd services
init.d services
Server Info
find: [REMOVED]: No such file or directory
Route Info
Netstat Info
Custom Files Info
Audit Generated in '50' seconds.
Submitting results to server using cURL
URL: https://[Actual.URL.of.Server]/open-audit/index.php/input/devices
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 294k 100 299 100 293k 11 11083 0:00:27 0:00:27 --:--:-- 0
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Unavailable</title>
</head><body>
<h1>Service Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
</body></html>
Audit Completed in '77' seconds.
I would check the Apache logs.
Per my original post, I've reviewed php, apache, and codeigniter logs. All the apache log has is a single line 503 error on the page request. It shows that the post submission size matches the expected size of the xml file being submitted.
The problem seems to be that open-audit is receiving the xml file and having some kind of issue processing it, but isn't logging an error anywhere. Is there anywhere to enable debug / verbose logging in open-audit beyond just the codeigniter logging?
Attach your result file to an email to marku@opmantek.com and I'll take a look.
The result of using create_file=y should remove the data= characters from the start of the file. Please check and advise if it is not.
I have just tested on Centos 6 and Redhat 7, both via discoveries and manually running it on the target machines. Both work as intended. Can you try running the below and posting the console output, please. Obviously replace YOUR_SERVER with your Open-AudIT servers resolvable name or IP.
./audit_linux.sh submit_online=y create_file=n url=http://YOUR_SERVER/open-audit/index.php/input/devices debugging=5
Mark.
We upgraded to 3.3.2 back in May. The server started up and displays data fine - everything appeared to be working. It took several weeks for me to notice that the weekly audit scripts (we run audit scripts manually against our linux environment using Rundeck) were failing.
Took a good chunk of time to narrow down what is happening, and I'm still not 100% sure where it's breaking down. I thought the issue was the submission from the audited systems, but at this point, even if I use the manual import and copy paste the contents of an xml file, I still wind up getting a 503 Service Unavailable error.
I've checked the php logs, the apache logs, and I've even enabled logging in codeigniter. Nothing is pointing to the source of the problem. It just tries to parse the xml file and seems to time out and die.
I'm kind of at my wits end trying to find where this is failing. Nothing shows up in any of the logs beyond the 503 message in the apache access log.
xml files being generated by the audit_linux.sh script on Red Hat 7 systems. Any help would be appreciated.