I have set up a bunch of alerts to run every 5min with a time range of the last 15min.
Every 5 min I get an email from the alert but when I run the search query it returns me ZERO events.
I did specified to only send emails of results > 0.
So I dont know why this is happening. Here is a screen shot of the email I receive showing (1) event !!??!? why?
Thanks
Try to update the alert to show the events inline and/or csv attachment in the email (select appropriate checkboxes in 'Alert Action' dialog). This way you'd see what Splunk is seeing to trigger alert.
Since your question is resolved, please accept somesoni2's comment/answer that led to the resolution.
Try to update the alert to show the events inline and/or csv attachment in the email (select appropriate checkboxes in 'Alert Action' dialog). This way you'd see what Splunk is seeing to trigger alert.
oh I see...if you look at my query it returns the results into a variable called "Occurences" . So even though results returned me ZERO, this option is counting that as a 1 event. Really? that is strange.
I didn't look at that. The stats count would give you a result with value of count as 0 if there are no rows. What you should do it to add a where clause in the query itself (e.g. ..| where Occurrences>0
) to check for count and then change the alert condition to trigger 'when number of events are greater than 0'.
As far as your base search is returning events stats count
will give default result as 0. Which implies that your alert will always be fired is the Trigger Conditions is Number of Results>0. As @somesoni2 mentioned, you either need to add final pipe as| search Occurences>0
to your alert search or else change your Alert Trigger conditions to Custom instead of Number of Results
and then set the Custom alert condition as | search Occurences>0
If you wish to show the trigger condition in your alert email, I would go with @niketnilay's suggestion of using custom trigger condition.
yeah that is what I did: search Occurences > xx
but by following your suggestion somesoni2 it pointed me out to the answer.