HI,
I would like to know if it's possible in the earliest / latest fields of a search to have something like:
index=myindex earliest="the time this search has started"-X seconds latest="the time this search has started" -Y seconds
Is it possible?
The aim is to run a saved search that populates a summary index, but I need to backfill this index with a search that contains earliest=-20h latest=-10h, but running the search as it is with the fill_summary_index.py command line returns no results because events I need to backfill occurs a long time before -20h (I need to backfill 4 month of datas).
I think I can have an eval statement that would compute what I want, and then just use where, but it would be very inefficient time wise.
Thanks,
EDIT
Thanks to the answers, I found that I can use earliest= [some search | return result] to populate the earliest field to look for data when I want. Problem is that I can't find a way to say:
earliest=["search that returns the starting scheduled time of this saved search"]
The keyword 'now' returns the starting time of the search when put inside the earliest field, but it's not what I want, I would like to have the starting SCHEDULED time, not the actual time I run the search.
I don't know if it's the right way to do it, and if there is another way, I would gladly try it.
Thank you very much martin for the help. I finally found the rest of the solution from here, use | addinfo, and info_min_time to retrieve the starting time of the search. So the final answer is (if you want earliest to start 20 hours before the scheduled time of the search eg):
index=_internal earliest=[ search index=_internal | head 1| addinfo | eval test=info_min_time-20*3600 | return $test]
EDIT
simplified, optimized, cleaned version:
index=_internal earliest=[ stats count | addinfo | eval test=relative_time(info_min_time, "-20h") | return $test]
Thank you very much martin for the help. I finally found the rest of the solution from here, use | addinfo, and info_min_time to retrieve the starting time of the search. So the final answer is (if you want earliest to start 20 hours before the scheduled time of the search eg):
index=_internal earliest=[ search index=_internal | head 1| addinfo | eval test=info_min_time-20*3600 | return $test]
EDIT
simplified, optimized, cleaned version:
index=_internal earliest=[ stats count | addinfo | eval test=relative_time(info_min_time, "-20h") | return $test]
Thank you very much, I have learned a lots of very valuable things on splunk today. It's been a loooong time since I want to find a way to have something evalued whithout pulling datas:
Never used relative_time before. It's really clean and less "hacky" than what I did. I update the answer
Since you're not actually using any data from _internal you could replace that with a call to stats or something else that doesn't cause any data to be loaded:
index=_internal earliest=[ stats count | addinfo | eval test=relative_time(info_min_time, "-20h") | return $test]
Additionally, you can let Splunk's relative time syntax do the time fiddling for you in case it gets more complicated.
If you can express your time fields using eval you can do a subsearch for each:
index=myindex earliest=[some search | eval earliest=something | return $earliest] latest=[some search | eval latest=something | return $latest] | ...
Isn't the time the search was run equal to now from the search's point of view?
nice! Another tip I didn't know.
I am still trying to find a way to get the search scheduled time start, but I didn't find it yet. Does scheduled search have a special field containing their scheduled time?
Your first attempt can be made to work like this:
index=_internal earliest=[ stats count | eval test="-24h" | return $test]
The dollar sign changes the behaviour of return, returning only the value itself instead of key=value as usual.
I just need to figure out how to get the time the search was scheduled, instead of the "now" time.
Peoblem solved, in fact result contained "test=-24h" and not only "-24h" as I was expected.
The correct search is then:
index=_internal [search index=_internal | head 1 | eval earliest="-24h" | return earliest]
which works wonderfully.
Many thanks for the help.
I have tried it but I couldn't make it works:
index=_internal earliest=[ search index=_internal | head 1 | return "-24h"]
, also tried:
index=_internal earliest=[ search index=_internal | head 1 | eval test="-24h" | return test]
but it always returns:
Error in 'search' command: Unable to parse the search: Comparator '=' has an invalid term on the left hand side.
THAT is awesome, didn't know you could run subsearches after an '=' !!!
Is it possible to run a subsearch like this: 'eval=[some subsearch]' ? It looks increadibly powerfull and will solve many performance problem that I have. Gonna test it right away.