I wanted to make the search faster. Is there any way?

[ryuch2002]

splunk enterprise version : 6.2.3

i using Django tag.

{% searchmanager
    id="nvd3_baseSearch3"
    search="index=\"jennifer\" sourcetype=\"jennifer_perf_agent\" tag=\"$server$\" | table hit_hour,hit_day,visit_day, visit_hour, agent, _time"|token_safe
    earliest_time="-7d@d"
    latest_time="now"
%} 

i using token,tag and earliest_time="-7d@d".

because of earliest_time="-7d@d", search is too late.

Is there any recommand way?

[MuS]

Hi ryuch2002,

there are multiple sources on docs and answers how to do it:

First identify the search you are doing, like is it a dense or sparse search - http://docs.splunk.com/Documentation/Splunk/6.2.3/Capacity/HowsearchtypesaffectSplunkEnterpriseperfo...

Next, try to adapt what's mentioned here http://docs.splunk.com/Documentation/Splunk/6.2.3/Search/Writebettersearches

and last but not least go through this answer http://answers.splunk.com/answers/172275/how-do-optimizations-for-field-based-searches-work.html and try to use all there techniques in your search.

@vganjare had a good approach, but on the wrong side of the | instead of using fields after the search, try to use as much fields as possible before the first pipe | like this:

index=\"jennifer\" sourcetype=\"jennifer_perf_agent\" tag=\"$server$\" hit_hour=\"*\" hit_day=\"*\"  visit_day=\"*\"  visit_hour=\"*\"   agent=\"*\" 

This is still not the best possible in regards of performance but a starting point.

cheers, MuS

[martin_mueller]

Say he has these two events:

  hit_hour  hit_day  visit_day  visit_hour  agent
1 12        20       20         12          firefox
2 13        21       21         13          <null>

If your search includes agent=* it will lose the second event.

Besides, filtering for agent=* will in 99% of cases not provide a performance boost because it doesn't change the events retrieved from disk and pushed through field extraction.

To get actual performance gains we need to know the requirements motivating this search. It's a huge list of events, does the user actually need millions of rows?

[MuS]

agreed - well, we provided some hints now its up to the questionnaire to use them 😉

[martin_mueller]

Uhm,no... it's not bad in regards of performance, it could be bad in regards of functionality.

[MuS]

Yes, it is for sure a matter of performance, because searches are faster when they use fields that have already been extracted (indexed fields) instead of fields extracted at search time. That said, if hit_hour="*" hit_day="*" visit_day="*" visit_hour="*" agent="*" are search time field extractions and it is a sparse search it will probably get worst.

Why do you reckon it is bad in regards of functionality?

[martin_mueller]

Be careful with all the field=* - just because he lists a field in the |table doesn't necessarily mean it exists in every event he's looking for.

[MuS]

That's why I have the last line in the answer 😉

[martin_mueller]

There is no benefit to specifying fields right before a table.

[vganjare]

You can specify fields command to avoid unnecessary field extraction.

search="index=\"jennifer\" sourcetype=\"jennifer_perf_agent\" tag=\"$server$\" | fields hit_hour,hit_day,visit_day, visit_hour, agent, _time | table hit_hour,hit_day,visit_day, visit_hour, agent, _time"|token_safe

Thanks!!