Splunk Search

Custom search command -> help

lpolo
Motivator

I have the phyton script presented in note 1. How Can I modify this script so it can be called as a splunk search command?

Note 1: uat_qe_feed.py

import urllib2, sys,csv
from lxml import etree
from time import strftime as date

host = ['host1.com','host2.com']

for s in host:

        url = "http://" + s + ":8080/rex/administration?files_used=true"

        try:

                f = urllib2.urlopen(url)
                doc = etree.XML(f.read())
                r = doc.xpath("//str[@name]")


                print(date('%Y-%m-%d %H:%M:%S') + "   " + "qe_host=" + s +  "   " + "Stack=uat" + "   " +  "LOCATION="+ r[1].text + "   " +  "NUMBER="+ r[2].text + "   " + "MAP="+ r[3].text + "   " +  "SET="+ r[4].text)


                f.close()

         except urllib2.URLError, e:

                print(date('%Y-%m-%d %H:%M:%S') + "   " + "qe_host=" + s + "   " + "Stack=uat" + "   " + "Status=QE_Not_in_Service" )
0 Karma

lpolo
Motivator

Script example customsearch.py:

import urllib2, sys,csv,time
from lxml import etree
import splunk.Intersplunk as si

host = ['host1.net','host2.net']
for s in host:

        url = "http://" + s + ":8080/rex/administration?files_used=true"
        #print(url)
        results = []
        now = str(int(time.mktime(time.gmtime())))

        try:
                f = urllib2.urlopen(url)
                doc = etree.XML(f.read())


                one =  doc.xpath("//str[@name='LINEUP']/text()")
                two  = doc.xpath("//str[@name='LINEAR']/text()")
                three  = doc.xpath("//str[@name='BITSET']/text()")

                results.append({'_time' : now,'qe_host' : s,'Stack' : 'cim','Status' : 'up','LINEUP' : one,'LINEAR' :  two,'BITSET' :  three})

                si.outputResults(results)



        except urllib2.URLError, e:
                results.append({'_time' : 'now','qe_host' : s,'Stack' : 'cim','Status' : 'down','LINEUP' : '','LINEAR' :  '','BITSET' :  ''})
                si.outputResults(results)

        f.close()

commands.conf
[customsearch]
filename = customsearch.py
generating = true
maxinputs = 1

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

I suspect you are fundamentally misunderstanding what a search command is good for. You appear to be trying to feed raw data into Splunk. Normally you would do this via a scripted input or simply a file, and index the data. But since a custom search command can run arbitrary code, it's expected that it outputs CSV field data. You can certainly just pass in raw text by putting it into a _raw CSV field, but it would make more sense if you also, at minimum, included _time in epoch time, as well as the other fields you already have available in Python. By putting it back into raw text line, it's wasteful, as you're simply forcing Splunk to re-parse fields that you've already parsed out.

lpolo
Motivator
0 Karma

sdaniels
Splunk Employee
Splunk Employee

Here are the steps for taking your python script and creating a splunk search command.

http://docs.splunk.com/Documentation/Splunk/latest/developer/searchscripts

Some examples referenced here for you to look at as well:

http://blogs.splunk.com/2011/11/30/using-custom-search-commands-with-splunk-python-sdk/

0 Karma

lpolo
Motivator

Thanks.

Solved by using as example:

/opt/splunk/etc/apps/search/bin/google.py

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

if you're looking for examples, several of the shipped Splunk search commands are in fact Python scripts. Look in $SPLUNK_HOME/etc/apps/search/bin and $SPLUNK_HOME/etc/apps/search/default/commands.conf.

0 Karma

lpolo
Motivator

I am getting the result set from a REST API call as shown in the script I presented. I am not indexing the result set.
I just need to know from this script example, how to converted to a custom search command.

0 Karma

lpolo
Motivator

Get the result set of an xml file. I have the script I presented in the initial post. I am able to print the results. I need to get these results from splunk by executing the script as a custom search. For example:

|uat_qe_feed

The result of this search command is the result of the last line of my phyton script:

print(date('%Y-%m-%d %H:%M:%S') + " " + "qe_host=" + s + " " + "Stack=uat" + " " + "LOCATION="+ r[1].text + " " + "NUMBER="+ r[2].text + " " + "MAP="+ r[3].text + " " + "SET="+ r[4].text)

What should I add in my code so this script can be called as a search command

0 Karma

sdaniels
Splunk Employee
Splunk Employee

What are you trying to accomplish with your custom search command? That may help us get you what you need.

0 Karma

lpolo
Motivator

i knew about the links and I was not able to make it work. I just need a single example from the code I presented to have a start up..

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...