Why does my python scripted input not work?

[andrewtrobec]

Hello,

I am trying to get a very simple python scripted input to work, but I cannot. I am using Splunk 6.6.3 running on Windows 10. The code for the scripted input named test.py is:

import sys
from time import localtime,strftime
import time

def main(argv):
    indexTime = strftime("%m/%d/%Y %H:%M:%S %p %Z",localtime())
    print "%s, name=%s,location=%s" % (indexTime,"Andrew","Europe")

if __name__ == '__main__':
    main(sys.argv[1:])

The file is located in $SPLUNK_HOME\etc\apps\search\bin

The inputs.conf located in $SPLUNK_HOME\etc\apps\search\local looks like this:

[script://$SPLUNK_HOME\etc\apps\search\bin\test.py]
disabled = false
index = main
interval = 60.0
sourcetype = csv

Now that it has been configured, nothing seems to happen. I don't even know whether the script is being run. When I run index=main as a search, it returns no results. index=_internal test.py doesn't show any signs of life either.

Could somebody help me troubleshoot? Is it a permissions issue? Is it the script? Is it the configuration?

Any point in the right direction would be greatly appreciated.

Best regards,

Andrew

[Yunagi · Accepted solution]

I suspect that the "csv" sourcetype is to blame. Your output does not have a header line.

Try:

[script://$SPLUNK_HOME\etc\apps\search\bin\test.py]
...
sourcetype = test

If you want to use the "csv" sourcetype, change the output of your script to print something like this:

print "date,name,location"
print "%s,%s,%s" % (indexTime,"Andrew","Europe")

[andrewtrobec]

This was it! Thank you so much!

[Yunagi]

You're welcome. 🙂

[damien_chillet]

Hi Andrew,

To start with,
Have you restarted Splunk after adding your scripted input?

[andrewtrobec]

I made sure to restart after every change. From another response it turns out that I was using the wrong sourcetype. Thanks for your input!

[sandyIscream]

First run the script from the folder and check whether it is giving you output or not.

Probably you have given wrong "\" ...replace this with "/"

[script://$SPLUNK_HOME/etc/apps/search/bin/test.py]
disabled = false
index = main
interval = 60.0
sourcetype = csv

Also check the user with which you have installed Splunk have permission to run the script.

[andrewtrobec]

I ran the script from another folder and ensured that all was working perfectly before moving it to the bin directory. When I deploy the app to a linux box then I'll be sure to remember the slashes. Thanks for your input!

[niketn]

@andrewtrobec, Refer to following documentation to write a basic python scripted input to Splunk: https://sublimerobots.com/2017/01/simple-splunk-scripted-input-example/

Add code to write to splunk's internal log using sys.stderr.write():

sys.stderr.write("python script is starting up\n") 

Which can be accessed using the following query,

index="_internal" sourcetype="splunkd" source="*splunkd" test.py

Also, you should test out your script in the Python compiler of your choice whether it runs independently of Splunk or not.

FYI Python code snippet from blog mentioned above

# So we can run this scipt under python 2 or 3
from __future__ import print_function

import sys                      # for sys.stderr.write()
import time                     # for strftime
from datetime import datetime   # for datetime.utcnow()
import random                   # to provide random data for this example

sys.stderr.write("TA-SimpleApp python script is starting up\n")                  

# output a single event
print (str(time.time()) + ", username=\"agent smith\", status=\"mediocre\", admin=noah, money=" + str(random.randint(1, 1000)))

[andrewtrobec]

Thanks for the link, very useful. Also, the sys.stderr.write call to debug and perform logging is great!