Alerting

Receive Alert Email When condition met twice in a row

shangshin
Builder

Hi,
I have a saved search to run every 2 hours. When the number of events is more than one, an alert email is triggered. How can I make it to trigger alert email only when the same condition happened twice in a row?

e.g.

2 AM - A, B, C

4 AM - B, D, E

Alert email is triggered for event B at 4 AM

Thanks in advance!

Tags (2)
0 Karma

shangshin
Builder

Thank you. I think this should the solution. I will give it a try.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The alert runs every 2 hr and if for two consecutive run some host has CPU> 90 then only an alert should be raised. With your condition, if within one 2hr period CPU> 90 for 2 time (2 15 min period) then the alert will go off, which is not desired, hence comparing two periods. Or other options can be tried, conforming to @linu1988 solution.

Value >90|bucket _time span=2h|stats count by _time | stats count |where count > 1

linu1988
Champion

which ever may be the case. If you see the average CPU utilization is more that 90% more than a time then how is it difficult to calculate?

Value >90|bucket _time span=15m|stats count(Value) as count by host|where count > 1

0 Karma

shangshin
Builder

Let's say the search is to scan a list of servers and it returns any server which cpu utilization higher than 90%. Therefore, the count is different for each run.

0 Karma

linu1988
Champion

what do you mean by condition? Isn't same as the count of the event more than 1 in 4AM? It's pretty simple.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

One dirty workaround could be to modify the search to run search for both the period ( if search runs at 10:00 AM then it should include result for 8:00 AM to 10:00 AM, again run for 6:00 AM to 8:00 AM) and compare count.
E.g.
search1 earliest=-2h@h latest=@h ..other terms | stats count as count1 | appendcols [search search2 earliest=-4h@h latest=-2h@h|..other terms | stats count as count2] | where count1>0 AND count2>0

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...