Include events from 7pm today onwards till next day same time.

[nabeel652]

Hi All

Monitoring backups activity I need to start 7pm each night till same time next day. How can I give range in my search so that only those event are included that are captured after 7pm each day and continue real time monitoring till 7 pm next day. My current time range
earliest= -1d@d+19h latest=now
works fine from the next day but each day after 7pm it is actually including events from yesterday 7pm which I don't want. I need it to refresh 7pm each day and last till 7pm next day and so on... Any ideas?

[Richfez · Accepted solution]

What's needed isn't quite what my previous answer had. Instead, what we need is a shifting time window. When it's BEFORE 7 PM, we'll want to use the time period of yesterday's 7 PM to today's 7 PM. If it's after 7 PM, then use today's 7 PM to tomorrow's 7 PM.

In the below example I used very lengthy variable names to be clear and prefixed them with an "a_..." so they'd show up at the top of the event list while I was testing. I left them here so if you were testing this piece by piece, you can see what the values are easily as well too. You can change all that.

<my random base search> earliest=-2d@d 
| eval a_UseYesterdayPeriod=if(tonumber(strftime(now(), "%H"))<19,1,0) 
| eval a_OldestTime=case(a_UseYesterdayPeriod=1,relative_time(now(),"-1d@d+19h"),1==1,relative_time(now(),"-0d@d+19h"))
| eval a_NewestTime=case(a_UseYesterdayPeriod=1,relative_time(now(),"-0d@d+19h"),1==1,now())
| where _time>a_OldestTime AND _time<a_NewestTime

I'll explain.

My base search has an earliest=-2d@d just to make sure we include by default all the times that may be included (all the way from yesterday's 7 PM until now).

The first eval defines a_UserYesterdayPeriod to be either true (1) if it's before 1900 (7 PM). All the pieces of strftime and tonumber are documented elsewhere.

The second and third evals create our two new time fields. If we're using yesterday's period, it's the first piece. If we're using today's period, it's the second half of it. Case is documented elsewhere, but basically it's a string of ifs with answers. "myVal=case(if a, then A, if b, then B)", and always equals 1 so 1==1 gives a "default" option of a sort.

Lastly we filter out were our times inside the period we needed.

Give those a try. I also have one more, possibly far more efficiet way to do this that I thought of, but it'll take a bit more time to come up with it and in the meantime this should work.

[Richfez]

I think this is it. The only real inefficiency is that you are pulling back a bunch of extra events, then filtering them out. You could change the initial earliest to match your oldest earliest you want - -1d@d+19h to limit this, then it shouldn't be too bad. You'll notice your timeline will include the blank periods, but if you do anything like | timechart count or something, it just removes all that empty space and only shows you the parts with data.

FYI, the thought I had was to search for essentially "nothing", build my evals and my times, then use those in an append to append all your data. But you can't pass variables around into your appended data, so I couldn't find a way to do that. Still, I thought I'd write that in here somewhere just in case someone else thinks of a way.

[nabeel652]

Awesome!

But I think we don't need

| eval a_NewestTime=case(a_UseYesterdayPeriod=1,relative_time(now( ),"-0d@d+19h"),1==1,now())

just set latest time to now( ) and it will work. However, won't make any difference if you still use this line.

[Richfez]

You just want latest to not be now, but instead today's version of "before 7pm", like

index=* earliest=-1d@d+19h latest=@d+19h

Try that!

[nabeel652]

Thanks for the answer. It sort of solved half of my problem but every day after 7pm I want to include events from after 7pm that onwards. So latest @d+19 is helpful before 7pm but after 7pm it will be showing events from yesterday's backup activity whereas after 7pm I want to initialize everything again and start looking at events after 7pm. I think I need some dynamic timerange in earliest that is -1d@d+19h before 7pm but @d+19h after 7pm every day and the latest will remain "now".

[Richfez]

OK, we can do that too. I think I'll make two answers because they really solve two different problems. We can just leave this one as it is - it may help someone who has the simpler version of the problem.