I was trying to accelerate a search with join command(trying to accelerate two searches). When I look at the job inspector, it says the search is using summaries for the search. But I think only the first search is accelerated and the second search is executes normally because the job inspector doesn't show the subsearch using any summaries.
I also tried accelerating the searches separately, with the expectation that each search in the combined search (search with JOIN) would match with the individual accelerated search and use the summary for searching. However the combined search only uses the summary created by the first search.
Lets say if this is the structure of the search
index=myindex sourcetype=sourcetype1 | eval something =1| stats count(something) as count by somethingelse |JOIN somethingelse [search index=myindex sourcetype=sourcetype2|stats max(splunk_is_awesome) as max by somethingelse] | table somethingelse max count
In the above search
index=myindex sourcetype=sourcetype1 | eval something =1| stats count(something) as count by somethingelse
gets accelerated but the subsearch doesn't get accelerated
index=myindex sourcetype=sourcetype2|stats max(splunk_is_awesome) as max by somethingelse
Any thoughts ??
I know you're anonymizing your search here, but would restructuring so you don't need the join and accelerating the one combined search be an option?
e.g. given your search, what about this one?
index=myindex (sourcetype=sourcetype1 OR sourcetype=sourcetype2)
| eval something = if(sourcetype=="sourcetype1",1,null())
| eval splunk_is_awesome = if(sourcetype=="sourcetype2",splunk_is_awesome,null())
| stats count(something) as count, max(splunk_is_awesome) as max by somethingelse
Something I realized... assuming you only in fact only looking at sourcetypes and indexes, you could actually do a metasearch to get your results. (it wouldn't work with anything parsed at search time, but if you're only using default index-time fields like index, _time, sourcetype, source it's fair game)
That is an option and I'm trying it out now. However I was expecting that there would be a way to accelerate searches with JOINS.
Friends don't let friends use JOIN.
Technically, some searches with join
such as yours can be accelerated. However, that doesn't happen in the way you were looking for.
Splunk looks at your search and scans it for the first reporting command, that'd be your stats count(something)
before the join
. This prefix of your search is sent off for acceleration. Whatever comes after that is irrelevant to the accelerator, any search beginning with this prefix is now using this acceleration.
In order to let your subsearch be accelerated as well you would need to save the subsearch as a report and accelerate that. To avoid code duplication you can use the savedsearch
command to load the saved subsearch from the saved main search: http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/Savedsearch
To tell if it is accelerated, are you looking at the debug output at the top of the job inspector? That might just apply to the main search. Is there a huge time difference to notice?
I was only looking at the job inspector. Since there is a section in the debug output that starts with "[subsearch]" which didn't show anything like "using summaries for search", my assumption is that the subsearch is not using summaries for search.
The search is certainly faster than the normal search as the first search uses summaries for searching. I can do a quick test to see if the subsearch is really using the summaries by removing the summaries for the subsearch and check if search becomes slower.
I did a quick test by slightly modifying my subsearch so that it doesn't use the summary (if it is using) anymore.
Comparing the runduration field in the job inspector the modified search (which doesn't use the summary for the subsearch) took almost the same time as the search that used the summary for subsearch. So the subsearch was not using the summaries for acceleration.
>In order to let your subsearch be accelerated as well you would need to save the subsearch as a report and accelerate that.
I think I wasn't clear. I also tried accelerating each search separately(main search and subsearch). However when I try to run the whole search only the main search is accelerated (uses the summary created by the accelerated main search) and not the sub search (doesn't use the summary created by the accelerated subsearch).
When both the searches are executed individually, it uses the summaries for the search.