How do I use results from a search in my custom command?

[sjoerdcopier]

I'm trying to use data from a search in a custom command.

source | scrapy url=uri

This gives me the following error:

Error in 'scrapy' command: This command must be the first command of a search.

It works when I use it as follows:

| scrapy url="www.splunk.com"

How can I make it work in combination with my search index?

[MuS]

Hi sjoerdcopier,

the important thing is to import the splunk.Intersplunk module in your script:

import splunk.Intersplunk

and read the results from the search into your script:

myresults,dummyresults,settings = splunk.Intersplunk.getOrganizedResults() # getting search results form Splunk
     for r in myresults: # loop the results

This way your script can pick up fields from the previous search results and it should work as expected if your search results contain a field called url (just rename uri to url) or change your script to use uri instead of url.

The link posted by @somesoni2 provides useful information as well.

Hope this helps ...

cheers, MuS

[sjoerdcopier]

Thanks MuS for your awnser,
I can't seem to get this right. Could you be so kind to help me out with an easy example?

   import splunk.Intersplunk

    # GET DATA FROM SEARCH
    myresults,dummyresults,settings = splunk.Intersplunk.getOrganizedResults() 

    # ILL DO SOME FANCY STUFF HERE

    # SEND DATA BACK

Thanks for helping out here.

[MuS]

Take a look at the docs, as usual everything you need is in there 😉

This http://docs.splunk.com/Documentation/Splunk/6.4.2/Search/Customsearchcommandshape will show an excellent example of a custom command which will use the previous search results, do stuff with it and return something to Splunk.

cheers, MuS

[somesoni2]

Since your second search (| scrapy url="www.splunk.com")works, It seems like you've a custom "Generating" search command which is generating the results.
For a custom search command to work with your first syntax ( source | scrapy url=uri), it should be a non-generating command. See definitions of customer search commands here
http://dev.splunk.com/view/python-sdk/SP-CAAAEU2