I have a simple search:
sourcetype=iis sc_status=500
The search returns results. I saved the search as an alert. The alert is cron scheduled to run every minute (Earliest: -1m@m
, Cron Expression: */1 * * * *
). The only condition on the alert is that results must be greater than 0.
When I open the alert in search, it gives results. When I look at the jobs page, I clearly see it running the alert search. Further, the jobs page clearly shows that many of these entries have positive result counts. When I inspect the job, I see the alert settings all look valid and resultCount is indeed a positive number.
However, the triggered alerts page shows nothing - not a single entry there. So what am I missing? Any tips on how to troubleshoot something like this?
Hi @leejeason,
Did you happen to set up the "Add to Triggered Alerts" alert action, for this particular alert? Or other alert actions? If not, this might explain some of the behavior you're seeing. If the "add to triggered alerts" action is not enabled, then the alert triggering instances won't be listed on the "Triggered Alerts" page.
Here is some documentation that might help:
http://docs.splunk.com/Documentation/Splunk/6.3.1/Alert/Triggeredalertaction
http://docs.splunk.com/Documentation/Splunk/6.3.1/Alert/Reviewtriggeredalerts
Let me know if you have other questions or are still seeing this behavior after double-checking alert action configuration.
@frobinson_splunk
Hi @leejeason,
Did you happen to set up the "Add to Triggered Alerts" alert action, for this particular alert? Or other alert actions? If not, this might explain some of the behavior you're seeing. If the "add to triggered alerts" action is not enabled, then the alert triggering instances won't be listed on the "Triggered Alerts" page.
Here is some documentation that might help:
http://docs.splunk.com/Documentation/Splunk/6.3.1/Alert/Triggeredalertaction
http://docs.splunk.com/Documentation/Splunk/6.3.1/Alert/Reviewtriggeredalerts
Let me know if you have other questions or are still seeing this behavior after double-checking alert action configuration.
@frobinson_splunk
Yep, that was it! I simply missed that there was an explicit action to do that and just assumed they'd end up there automatically. Thanks so much for the clarification!
Oh, glad this helped!
Cheers,
@frobinson_splunk