Alerting

Receive Alert Email When condition met twice in a row

shangshin
Builder

Hi,
I have a saved search to run every 2 hours. When the number of events is more than one, an alert email is triggered. How can I make it to trigger alert email only when the same condition happened twice in a row?

e.g.

2 AM - A, B, C

4 AM - B, D, E

Alert email is triggered for event B at 4 AM

Thanks in advance!

Tags (2)
0 Karma

shangshin
Builder

Thank you. I think this should the solution. I will give it a try.

0 Karma

somesoni2
Revered Legend

The alert runs every 2 hr and if for two consecutive run some host has CPU> 90 then only an alert should be raised. With your condition, if within one 2hr period CPU> 90 for 2 time (2 15 min period) then the alert will go off, which is not desired, hence comparing two periods. Or other options can be tried, conforming to @linu1988 solution.

Value >90|bucket _time span=2h|stats count by _time | stats count |where count > 1

linu1988
Champion

which ever may be the case. If you see the average CPU utilization is more that 90% more than a time then how is it difficult to calculate?

Value >90|bucket _time span=15m|stats count(Value) as count by host|where count > 1

0 Karma

shangshin
Builder

Let's say the search is to scan a list of servers and it returns any server which cpu utilization higher than 90%. Therefore, the count is different for each run.

0 Karma

linu1988
Champion

what do you mean by condition? Isn't same as the count of the event more than 1 in 4AM? It's pretty simple.

0 Karma

somesoni2
Revered Legend

One dirty workaround could be to modify the search to run search for both the period ( if search runs at 10:00 AM then it should include result for 8:00 AM to 10:00 AM, again run for 6:00 AM to 8:00 AM) and compare count.
E.g.
search1 earliest=-2h@h latest=@h ..other terms | stats count as count1 | appendcols [search search2 earliest=-4h@h latest=-2h@h|..other terms | stats count as count2] | where count1>0 AND count2>0

Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...