Splunk Dev

How to output raw results from the Splunk Python SDK when using pagination?

reswob4
Builder

So I was having the same problem as here:

https://answers.splunk.com/answers/99633/only-100-results-return-with-python-api-query.html

and I used this: http://dev.splunk.com/view/SP-CAAAEE5#paginating

and this: http://dev.splunk.com/view/python-sdk/SP-CAAAER5

To create a script that now returns ALL my results, BUT I need to write the raw results to a file.

I WAS using the following syntax:

<snip>
kwargs_normalsearch = {"exec_mode":"normal"}
job = service.jobs.create(searchquery, **kwargs_normalsearch)

<check for job to be done>

content = str(job.results(output_mode="raw"))
file.write(content)
<snip>

But I have not figured out how to combine the output_mode="raw" into the pagination script. I have not yet found anything in the above documentation or via Google showing the correct syntax, so I'm asking here.

I basically copied what was on one of the pages listed above:

kwargs_blockingsearch = {"exec_mode":"blocking"}

print "Search results:\n"
resultCount = job["resultCount"]  # Number of results this job returned
offset = 0;                       # Start at result 0
count = 10;                       # Get sets of 10 results at a time

while (offset < int(resultCount)):
    kwargs_paginate = {"count": count, "offset": offset}

    # Get the search results and display them
    blocksearch_results = job.results(**kwargs_paginate)

    for result in results.ResultsReader(blocksearch_results):
        print result

    # Increase the offset to get the next set of results
    offset += count

And I've tried:

blocksearch_results = job.results(**kwargs_paginate, output_mode="raw")

and

blocksearch_results = str(job.results(**kwargs_paginate, output_mode="raw"))

and

kwargs_paginate = {'output_mode": "raw", "count": count, "offset": offset}

all fail due to invalid syntax.

Does anyone have any suggestions?

1 Solution

reswob4
Builder

I figured it out.

Once I realized the results are returned in ordered dictionaries, here's what i did:

    searchquery = "search <your search query> | table _raw"
.....
.....
    for result in results.ResultsReader(blocksearch_results):
          event=result.popitem()
          print event[1]

This returns only the _raw results for each. I'm writing them to a file

f.write(event[1] + "\n")

do with them what you will.

View solution in original post

reswob4
Builder

I figured it out.

Once I realized the results are returned in ordered dictionaries, here's what i did:

    searchquery = "search <your search query> | table _raw"
.....
.....
    for result in results.ResultsReader(blocksearch_results):
          event=result.popitem()
          print event[1]

This returns only the _raw results for each. I'm writing them to a file

f.write(event[1] + "\n")

do with them what you will.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...