I see significant search time discrepancy when I run a one-shot search via the python SDK as opposed to when I run the same search in Splunk web with the same user, has anyone one else seen this? I assume I'm doing something wrong.
I am using the basic code form the example found: http://dev.splunk.com/view/python-sdk/SP-CAAAEE5
The only changes I added were in the kwargs_normalsearch I added earliest_time": "-60m ,
"latest_time": "now" .
It works just fine, but it is super slow. Me running the search in the GUI 46 seconds. Me running the search via REST API, 5 minutes plus. Not sure what the issues are???
Any guidance will be appreciated, Thanks in advance.
CODE:
import sys
from time import sleep
import splunklib.results as results
...
Initialize your service like so
import splunklib.client as client
service = client.connect(username="admin", password="yourpassword")
searchquery_normal = "search * | head 10"
kwargs_normalsearch = {"exec_mode": "normal"}
job = service.jobs.create(searchquery_normal, **kwargs_normalsearch)
A normal search returns the job's SID right away, so we need to poll for completion
While True:
while not job.is_ready():
pass
stats = {"isDone": job["isDone"],
"doneProgress": float(job["doneProgress"])*100,
"scanCount": int(job["scanCount"]),
"eventCount": int(job["eventCount"]),
"resultCount": int(job["resultCount"])}
status = ("\r%(doneProgress)03.1f%% %(scanCount)d scanned "
"%(eventCount)d matched %(resultCount)d results") % stats
sys.stdout.write(status)
sys.stdout.flush()
if stats["isDone"] == "1":
sys.stdout.write("\n\nDone!\n\n")
break
sleep(2)
# Get the results and display them
for result in results.ResultsReader(job.results()):
print result
job.cancel()
sys.stdout.write('\n')
SEARCH:
searchquery_normal = '''| tstats summariesonly=t chunk_size=10000 count first(All_Traffic.action) as action first(All_Traffic.icmp_type) as icmp_type from datamodel=Network_Traffic by _time span=1s All_Traffic.dest_ip All_Traffic.src_ip All_Traffic.dest_port All_Traffic.src_port All_Traffic.transport | `drop_dm_object_name("All_Traffic")` | eval external_src=if((!cidrmatch("10.0.0.0/8",src_ip) AND !cidrmatch("REDACTED/12",src_ip) and !cidrmatch("192.168.0.0/16",src_ip)),"true", "false") | eval external_dest=if((!cidrmatch("10.0.0.0/8",dest_ip) AND !cidrmatch("REDACTED/12",dest_ip) and !cidrmatch("192.168.0.0/16",dest_ip)),"true", "false") | eval external_dest=if(cidrmatch("REDACTED/16",dest_ip) AND external_src="false", "false", external_dest) | eval external_dest=if(cidrmatch("REDACTED",dest_ip) AND cidrmatch("REDACTED/16",src_ip), "false", external_dest) | search external_dest="true" OR external_src="true" | fields - external_*'''.format(value_to_check, value_to_check)
... View more