Currently, we have a search that is set to trigger if it returns a single result, and then throttle for 10 minutes before going again.
We'd like to kind of do the opposite: If the search is STILL returning results (same host OR other hosts) after 10 minutes' time, THEN trigger an alert.
At the moment, this search returns alerts as soon as they happen, but sometimes it's a single alert and therefore a minor warning, and sometimes it's continuous (aka a service is actually down). We'd like to get Splunk to trigger if the same alert is still firing after 10 minutes, which usually indicates a problem with a particular host.
... View more