Alerting

How to send alert based on number of occurrences by using email trigger

subramanianp789
New Member

Hi,
I have an requirement that need to schedule the below search query for every 2 mins(it can be given in corn schedule */2 * * * *) but it should not trigger mail immediately even breach threshold and the after specified time limit which will be mentioned in the lookup csv file, consider field as "count_threshold"=3

Now, after 6 mins (ie 2 mins * 3 = 6 mins) the email has to trigger if the search query breach the threshold(or if the results greater than 0)

Below is the existing search:

| eval "Alert Status" = case((' Virtual Bytes'<=manual_threshold3),"NORMAL", (' Virtual Bytes'>manual_threshold4),"CRITICAL", (' Virtual Bytes'>manual_threshold3 AND ' Virtual Bytes'<=manual_threshold4),"WARNING")
| search "Alert Status"="CRITICAL"
| table  Host," PID","Process Name"," Virtual Bytes","Alert Status"

How to achieve this? And how to add the condition to send mail after 6 mins.

Thanks.

0 Karma

woodcock
Esteemed Legend

The way to do this is to write out the results of this search to a lookup file with ... | outputlookup. Then create another scheduled search that runs on a different cron schedule that mines the outputs of the first search using | inputlookup ... and applies your throttling/email logic. This second search (or even a third search) can trim/cleanup the lookup so that it doesn't grow out of control in size.

0 Karma

subramanianp789
New Member

Thanks a lot for your valuable comments. Can you please explain more in detail about the inputlookup. It would be great if you provide me some sample steps to proceed further. Thanks again.

Also, how to pass the value from lookup csv to cron schedule or trigger condition, where to include. Please share your thoughts on this

0 Karma

woodcock
Esteemed Legend

My answer gives all the basic pieces; how you put them together is up to you. The main thing is to put as much of your thresholding and throttling logic inside of the search SPL as possible (try to keep the alert action number of results and is greater than 0).

0 Karma
Get Updates on the Splunk Community!

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...