Alerting

How to create an "OK" alert - after an alert has been issued?

JYTTEJ
Communicator

I have created this SPLUNK alert, which is logged as below:

Oct  4 19:03:46 psdkxs01 alert-splunk: MID=SxxI002E;DAT="xxxxxxxx timeout"

The alert is triggered when x number of MID=CxxI002W has been issued within the last 10 minutes.

Now I want to create an 'OK' alert on following conditions:

If MID=CxxI002W has not been issued for the last 10 minutes - AND MID=SxxI002E has been issued within the last 20 minutes - the OK alert has to be issued.

The logs for MID=CxxI002W looks like this:

Oct  4 19:06:01 psdkxp05 MID=CxxI002W;;STM=2011-10-04 19:06:01,475

How do I accomplish this?

Tags (1)

Damien_Dallimor
Ultra Champion

Maybe try something like (substitute in your own sourcetype) :

sourcetype=foo earliest=-10m@m MID=CxxI002W |  stats count as a | join  [search sourcetype=foo earliest=-20m@m MID=SxxI002E | stats count as b ] 

Then set the Alert condition to "if custom condition is met"

And set the "Custom condition search" to be :

search a = 0 AND b > 0

sbsbb
Builder

Thats a nice workaround, but in that case we have to configure 2 Search for each thing beeing monitored...

Splunk need some kind of ok condition to fire an "all ok" alert, and remove the entry in "alerts" this is missing

Damien_Dallimor
Ultra Champion

cool...don't forget to "accept" the answer...helps out splunkbase.

0 Karma

JYTTEJ
Communicator

GREAT - thank you! I did try something similar with the earliest=-10m and earliest=-20 minutes - but got stuck with an error message. You have solved my problem.

0 Karma
Get Updates on the Splunk Community!

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...