TenableSC Modular Input error

[michael_reeves]

I have never been able to get the TenableSC Modular Input to work. I get the following error:

<error><message>Error Querying Security Center: Error performing vuln::query::vulndetails : No JSON object could be decoded</message></error>

Because this error is occurring when running the query, the JSON for authenticating and getting a token work. The actual JSON which errors out looks like this

 {'request_id': '1', 'action': 'query', 'module': 'vuln', 'token': 123456789, 'input': '{"sourceType": "cumulative", "tool": "vulndetails", "startOffset": "0", "endOffset": "54321"}'}

The only problem with the JSON I can see is the use of ' instead of ". However, I cannot explain why the double-quotes are being replaced with single-quotes.

Has anyone been able to get this modular input to work or found another way to import Tenable Security Center data into Splunk?

[wwatson8351]

Wanted to share my experience with this app after troubleshooting it with Mr bradp123 (thank you btw!)

*Everything i'm writing was done on Splunk 6.2*

First, the python script needed to be adjusted for me. For some reason it was passing 4 arguments when i was only giving it three (username, password, url). In order to fix this i changed the main portion to look like this: `

if __name__ == '__main__':
        if True:
                username=sys.argv[1]
                password=sys.argv[2]
                url=sys.argv[3]

                sc = sc_connect(username, password, url)
                sc.vulnipdetail()
        else:
                print "Usage $SPLUNK_HOME/bin/splunk cmd python %s \"username\" \"password\" \"url\"" % sys.argv[0]
                print "\n\n"
                print "Where:-"
                print "    username             is a valid Security Center Username"
                print "    password             is the password for the Security Center Username"
                print "    url                  is a valid Security Center request URL, example:- https://192.168.1.2/request.php";
                print "\n\n"
                print "Running this script directly is for testing purposes only."
                print str(len(sys.argv))

Second, i needed to add the proper sourcetype to the props.conf file and changed the regex for the LINE_BREAKER. Here is what this looks like for me:

[security_center_vulndetails]
BREAK_ONLY_BEFORE=(pluginID)
KV_MODE=json
#LINE_BREAKER=(,|\[)\{
LINE_BREAKER=\}(,)\{
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true
TRUNCATE=1000000

[bradp123]

Hello,

The reason you are unable to get any data is because you are trying to pull too many results from Security Center at once and the SC API is unable to return that many results. The original app creator must have been working with a very small amount of data. I ran into the same error mesage with my endoffset being over 70,000.

To fix this issue, you have to reduce the total records being pulled at one time. I modified the /opt/splunk/etc/apps/tenablesc/bin/sc_connect.py file to pull data in chunks of 5,000. This got my data into splunk. The only downside that I have not fixed is that certain records are not parsed correctly due to additional info being returned (e.g. error_code and timestamp). Here is the code I modified in the sc_connect.py file:

def vulnipdetail(self):
             try:
                     #the first query returns the first 1000 records and the total number of records
                     start, end = 0,1000
                     input = {"tool": "vulndetails", "startOffset": "0",
                              "endOffset": end ,
                              "sourceType": "cumulative"}

                     inputjson = json.dumps(input)

                     data = {"request_id": "1",
                             "module": "vuln",
                             "action": "query",
                             "input": inputjson,
                             "token": self.token}
                     response, content = self.HttpRequest(data)
                     #print the first 1000 records
                     print content
                     result = json.loads(content)
                     total = result['response']['totalRecords']
                     #this loops through the rest of the records and pulls them in chunks of 5000
                     while end < int(total):
                             start, end = end, end + 5000
                             input = {"tool": "vulndetails", "startOffset": start, "endOffset": end, "sourceType": "cumulative"}
                             inputjson = json.dumps(input)
                             data = {"request_id": "1", "module": "vuln", "action": "query", "input": inputjson, "token": self.token}
                             response, content = self.HttpRequest(data)
                             #I tried to concat all the content, but the stdin buffer truncated the results. This causes some records to not be parsed correctly. 
                             print content

              except Exception, e:
                     raise Exception, "Error performing vuln::query::vulndetails : %s" % str(e)

[mcluver]

Excellent work! Thank you.