Splunk Search

What Hunk search language command generates a map reduce job in hadoop cluster?

splunkIT
Splunk Employee
Splunk Employee

What search commands in Hunk kick off reducers vs. trying to collection data via a streaming session? I ask, since I looked at the search log for a query of search index=vir-test minutesago=60.

06-17-2015 21:58:42.616 INFO ERP.isa-prod - SplunkMR$SearchHandler - Reduce search: null 
06-17-2015 21:58:42.617 INFO ERP.isa-prod - SplunkMR$SearchHandler - Search mode: stream 
06-17-2015 21:58:42.617 INFO ERP.isa-prod - SplunkMR$SearchHandler - setting requiredFields=* 

Based on the data, it appears that a streaming job was kicked off (not too fast). I have looked at
http://docs.splunk.com/Documentation/Hunk/6.2.3/Hunk/distributableandnondistributablesearchcommands , but it isn't clear as to which commands kick of a reducer.

splunkIT
Splunk Employee
Splunk Employee

A map-only MR job will be submitted to Hadoop when the search a) contains any reporting / transforming commands (assuming verbose mode is not in use) or b) the search contains filtering predicates

http://docs.splunk.com/Documentation/Splunk/6.2.3/Search/Aboutreportingcommands

For example:

index=vir-test <<-- that's just streaming data

index=vir-test error OR warn <<-- this should kick off a MR job

index=vir-test | stats count by my_field <<-- this should kick off a MR job

ddrillic
Ultra Champion
index=vir-test error OR warn

is intriguing.

I tried -

index=xxxx  source = "*part-m-00078*" OR source = "*part-m-00079*"

I see the MapR job running, but the query runs at a very slow speed.
Weird thing.

0 Karma

tsunamii
Path Finder

RE: A map-only MR job will be submitted to Hadoop...
So Hunk will never kick-off a reducer job on the hadoop side?

0 Karma

pburgu_splunk
Splunk Employee
Splunk Employee

That's correct. The reduce function happens on Hunk search head.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...