Alerting

Alert based on evaluating two time ranges

nbharadwaj
Path Finder

I am trying to send alerts based on the following criteria:

Last 15-20 mins: Measure avg(field1) --> value1 Last 10-15 mins: Measure avg(field1) --> value2

Calculate the % change between value1 and value2:

diff = (value2-value1)*100/value1

If diff is more than X percent than I need to send an alert notification.

How can I do this in one search command?

1 Solution

sideview
SplunkTrust
SplunkTrust

In the TimeRangePicker, choose 'custom time', then go to 'advanced'.

Enter the following as your timerange.

earliest = -20min@min

latest = -10min@min

And here's a search that will work:

<your search terms> | eval period=if(_time<relative_time(now(), "-15min@min"),"older_period", "recent_period") | eval foo=1 | chart avg(field1) over foo by period | eval diff = (recent_period-older_period)*100/older_period | where diff>X

Then save the search, click the 'schedule this search' box, and tell splunkd that if the search returns 1 or more rows, that you want it to send you an email.

here's another different search that will work too, although you have to use the same timerange

<your search terms> | timechart avg(field1) as avg span=5min | transpose | search column=avg | rename "row 1" as "older_period" | rename "row 2" as "recent_period" | eval diff = (recent_period-older_period)*100/older_period | where diff>X

View solution in original post

sideview
SplunkTrust
SplunkTrust

In the TimeRangePicker, choose 'custom time', then go to 'advanced'.

Enter the following as your timerange.

earliest = -20min@min

latest = -10min@min

And here's a search that will work:

<your search terms> | eval period=if(_time<relative_time(now(), "-15min@min"),"older_period", "recent_period") | eval foo=1 | chart avg(field1) over foo by period | eval diff = (recent_period-older_period)*100/older_period | where diff>X

Then save the search, click the 'schedule this search' box, and tell splunkd that if the search returns 1 or more rows, that you want it to send you an email.

here's another different search that will work too, although you have to use the same timerange

<your search terms> | timechart avg(field1) as avg span=5min | transpose | search column=avg | rename "row 1" as "older_period" | rename "row 2" as "recent_period" | eval diff = (recent_period-older_period)*100/older_period | where diff>X

Get Updates on the Splunk Community!

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...