I am trying to create a search that looks at the average number of errors over the past 4 weeks, but only looking at the days that match the current day. (I am using the past 4 Mondays as an example, I have created a macro to use in the search string to get the actual current day).
Currently I am just using date_wday=monday
in the initial search string, and setting the time range to the past however many weeks. This seems to work perfectly if I select the past two weeks, the job runs in about 2-4 seconds. However if I set the time range to the past 3 weeks or more, the search takes 300+ seconds.
I would have no idea what would be a more efficient way of performing this task, the exact same issue occurs when I use multiple earliest
/latest
to achieve the same thing.
If anyone knows what could be causing this, please let me know!
| tstats count as w_count where index=your_index TERM("error") by _time
| eval date_wday=strftime(_time,"%w")
| where date_wday = 1
| stats avg(w_count)
It is fast if error
is correct.
The number that contains the strings of error
in the log is counted.
Hi,
I have seen something similar if using cheaper storage for cold buckets. It would greatly reduces speed if the storage is slower. Also, how many events are you talking about? How many events in two weeks and what is the difference for 3 weeks?
Using the date_wday is a bit tricky as not all events will have that field. This is created only when you have the timestamp on the event and it also doesn't change timezones. For more information about it, check here: https://docs.splunk.com/Documentation/Splunk/8.0.1/Knowledge/Usedefaultfields
You might also be thinking about data model acceleration if you want better performance on searches over large sets of data and specially over large amounts of time.