Qualys Technology Add-on (TA) for Splunk: Does the forwarding or parsing process run at scheduled intervals?

[pkeller]

I have the Qualys Technology Add-on (TA) for Splunk configured and it is collecting data from Qualys ... but that data, in xml format, is just sitting in TA-QualysCloudPlatform/tmp and I'm not seeing it forwarded to Splunk

So ... assuming that the add-on has some built-in mechanism for parsing the collected XML and then forwarding to Splunk ...

1. Does that forwarding/parsing process run at scheduled intervals?
2. Does it remove the collected files once they've been forwarded?
3. What process is actually reading and forwarding the collected data?
4. Is there a way to configure a higher level of logging for this application? There's nothing very useful being written to the logs under var/log/splunk with regards to this TA. (so we didn't find out that our authentication was failing until long after it started)

Thank you.

[pkeller]

And with regards to the tmp directory ... There is a readme ...

[splunk@somwehere tmp]$ cat readme.txt
Files in this directory are transient and should get deleted after parsing of XML is done, if you see old files in these directories lying around for a long time, you can safely delete them.Empty directory to hold api output temporarily.

Currently using version 1.2.3 of TA-QualysCloudPlatform

[pkeller]

I obviously posted this a long time ago and embarrassingly neglected to comment on the provided answer. It was so long ago that I don't recall the specifics about it and we clearly resolved whatever issues we were having. Apologies to nit123 for the lateness in my reply.

[TonyLeeVT]

I have the same issue... The data was downloaded in the tmp directory and is just sitting there. Did you ever figure out what comes along to parse and clean up the data?

[nit123 · Accepted solution]

Can you confirm the version of TA please ?

To share more light on your questions, refer to the following

Does that forwarding/parsing process run at scheduled intervals?

For each input enabled, Splunk calls the modular input every minute. With each call, it passes XML containing information regarding that input. You need to configure supported inputs before you can use them. Go to Settings >Data Inputs > TA-QualysCloudPlatform and click New button. Enter asked inputs, and click Next. Again go to Settings > Data Inputs > TA-QualysCloudPlatform, and enable required input(s).

Does it remove the collected files once they've been forwarded?

If you are talking of the files in /tmp folder, note that when the data is being fetched the API responses are stored in XML files that are later deleted when TA is done with processing. XML files are meant for temporary parsing workflow.

What process is actually reading and forwarding the collected data ?

Once the TA is configured and data inputs are enabled, the TA will start fetching the data from Qualys API server. When TA runs for any data input (host detection/knowledge base/WAS), it creates a PID file per input, and writes the current process' PID in it. XML files in tmp directory contain the API response and that is parsed by threads of TA and then data is indexed to Splunk under the index you selected. Note that if you have set your splunk instance as forwarder to X IP, the data shall be forwarder to the indexer without saving on the forwarder (Light forwarder). If its a heavy forwarder, data is saved and also forwarded.

Is there a way to configure a higher level of logging for this application ?

TA write logs for error conditions on every milestone of TA. There seem to be a lot of improvements into logging & error handling gone into the newer release of TA and that shall serve user with detailed information on what is possibly wrong with TA processing if at all.

Hope this write-up offer you some assistance. Thanks.

[TonyLeeVT]

While this adds some nice context to the questions, it does not actually answer the specific questions. Can you please be more specific in your answers?