Splunk Search

How to alert on a value crossing over a moving average of that value?

responsys_cm
Builder

Is there any way to do this in a single search? I know it can be done by having one search compute the moving average of the field and then write that to a lookup table. A subsequent search would then compute the value for the time period and see if it is above the value in the lookup table.

Is there any way to do it with a single search that doesn't require a lookup table?

Thx.

Craig

Tags (2)
0 Karma

mhamano
Explorer

I'm trying to run a similar search looking at decline rates for a payment gateway. I want to look at the current decline rate average trend line and also the running average trendline.

tried doing something similar but ended up getting Mismatched ']' when I ran my query.

gateway=firstdata errorType=declined event=transaction Completed earliest=-1h@h latest=@h | bucket _time span=1h | stats count as hourlyCount by _time | appendcols { search searchforerrors earliest=-7d@d latest=-1h@h | bucket _time span=1h | stats count by _time | stats avg(count) as average ] |where hourlyCount > average

Any idea why?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

actually trendline or the streamstats commands will let you do this fairly easily, but really it's hard to say what works for you unless we know how you are averaging your data.

lguinn2
Legend

Let me make this more specific, so that i can show an example. Every hour, I want to compare the number of errors from the past hour with the average number of errors for the past week. If the hourly count > the average count, then trigger an alert.

This search will do

searchforerrors earliest=-1h@h latest=@h
| bucket _time span=1h
| stats count as hourlyCount by _time
| appendcols { search  searchforerrors earliest=-7d@d latest=-1h@h
      | bucket _time span=1h
      | stats count by _time
      | stats avg(count) as average ]
| where hourlyCount > average

Schedule this search to run once each hour and to trigger if the number of results is greater than zero.

mhamano
Explorer

I'm trying to run a similar search looking at decline rates for a payment gateway. I want to look at the current decline rate average trend line and also the running average trendline.

tried doing something similar but ended up getting Mismatched ']' when I ran my query.

gateway=firstdata errorType=declined event=transaction Completed earliest=-1h@h latest=@h | bucket _time span=1h | stats count as hourlyCount by _time | appendcols { search searchforerrors earliest=-7d@d latest=-1h@h | bucket _time span=1h | stats count by _time | stats avg(count) as average ] |where hourlyCount > average

Any idea why?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...