All Apps and Add-ons

splunk: nessus2splunk.py [unable to get data in] and [err: $SPLUNK_HOME may not be set]

tkropp
Path Finder

We are currently testing the Splunk_TA_nessus and attempting to add Nessus data.

Running the defaults yields the following message in the splunkd.log

12-08-2014 17:10:36.665 -0500 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus2splunk.py -s /opt/splunk/etc/apps/Splunk_TA_nessus/spool -t /opt/splunk/var/spool/splunk

We have tried to run the python script from within splunk:

/opt/splunk/bin/splunk cmd /opt/splunk/etc/apps/Splunk_TA_nessus/nessus2splunk.py

This also yields no results.

We are using the defaults in the README for the TA and attempting.

Example of use in inputs.conf:

[script://./bin/nessus2splunk.py]
disabled = false
interval = 120
index = _internal
source = nessus2splunk
sourcetype = nessus2splunk

Example of use in inputs.conf using custom source and target directories
for input and output files:

[script://./bin/nessus2splunk.py -s /opt/nessus/incoming -t /opt/nessus/parsed]
disabled = false
interval = 120
index = _internal
source = nessus2splunk
sourcetype = nessus2splunk

Also, when specifying the target directory, the following error is in the log

12-08-2014 17:41:35.753 -0500 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/Splunk_TA_nessus_ES/bin/nessus2splunk.py
12-08-2014 17:41:35.753 -0500 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/Splunk_TA_nessus_ES/bin/nessus2splunk.py -s /data/nessus/in -t /data/nessus/out
12-08-2014 17:41:36.658 -0500 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_nessus_ES/bin/nessus2splunk.py" usage: nessus2splunk.py [-h] [-s SRCDIR] [-t TGTDIR]
12-08-2014 17:41:36.658 -0500 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_nessus_ES/bin/nessus2splunk.py" nessus2splunk.py: error: argument -s/--srcdir: Invalid path specified ($SPLUNK_HOME may not be set).

0 Karma
1 Solution

tkropp
Path Finder

Working together we have solved the problem. (our particular problem).

1) The native nessus file format located in /opt/nessus/var/nessus/users/admin/reports is the DATABASE format (binary encoded) and therefore not consumable by the Splunk-TA.

2) Once exported from the GUI into the .nessus (v2) XML format, the nessus2splunk.py script works like magic.

3) We were then challenged with how (?) to automate (via Nessus API) extraction of reports so that we don't have to use (ugh) a GUI. Then splunk can do its thing via inputs.conf.

Our solution? Use a python script to access the Nessus API, then wrap that script to handle file movements (between splunk running as splunk, and nessus running as root) and associated api calls

https://github.com/djfang/py-nessus-wrapper which uses ("wraps") https://github.com/metaevolution/py-nessus

View solution in original post

tkropp
Path Finder

Working together we have solved the problem. (our particular problem).

1) The native nessus file format located in /opt/nessus/var/nessus/users/admin/reports is the DATABASE format (binary encoded) and therefore not consumable by the Splunk-TA.

2) Once exported from the GUI into the .nessus (v2) XML format, the nessus2splunk.py script works like magic.

3) We were then challenged with how (?) to automate (via Nessus API) extraction of reports so that we don't have to use (ugh) a GUI. Then splunk can do its thing via inputs.conf.

Our solution? Use a python script to access the Nessus API, then wrap that script to handle file movements (between splunk running as splunk, and nessus running as root) and associated api calls

https://github.com/djfang/py-nessus-wrapper which uses ("wraps") https://github.com/metaevolution/py-nessus

bradp1234
Path Finder

Hello,

When you integrated the script into your workflow, how did you do it? Did you use a cron job or utilize Splunk?

0 Karma

tkropp
Path Finder

Hi @bradp1234 we used a non-splunk cron job. The script should probably be added to the py-nessus-wrapper for Splunk.

A note on the wrapper, we are using Nessus 5, and have not attempted this on Nessus 6.

Here is the script to take the reports and change the ownership to splunk:splunk (needed if Splunk Enterprise is running as non-root)

##! /bin/bash

cd /opt/nessus/var/nessus/users/admin/reports.csm.orig

# Remove all current files

nessus_inj='/usr/bin/find -mtime -1 *.nessus.v1'
dest='/data/nessus/in'

# Remove old files
/bin/rm -rf $dest/.``

for i in $nessus_inj
do
cp $i $dest/
done

/bin/chown -R splunk:splunk $dest/*
exit 0

0 Karma

bradp1234
Path Finder

Great! Thanks for the info!

0 Karma

tkropp
Path Finder

fwiw we also tried this:

/opt/splunk/bin/splunk cmd python /opt/splunk/etc/apps/Splunk_TA_nessus/bin/nessus2splunk.py -s /data/nessus/in -t /data/nessus/out

And this didn't give any errors. It also didn't do anything with the data located in /data/nessus/in

(permissions are okay, files are okay)

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...