Splunk Search

How to count the number of events that are greater than 2 Standard deviations from the mean

richnavis
Contributor

I have events with response times.. and I want to find out how many of these events have response times > 2 Standard Deviations.

Tags (1)
0 Karma
1 Solution

melonman
Motivator

Hi

A bit messy, but is this what you are looking for?

sourcetype=your_sourcetype [search sourcetype=your_sourcetype | stats stdev(reponse_time) as stdev | eval search="response_time>".(stdev*2) | fields + search] | stats count

View solution in original post

emiller42
Motivator

There is another approach that doesn't require a subsearch:

sourcetype=your_sourcetype  | streamstats avg(response_time) as average stdev(response_time) as standard_deviation  | where response_time>average+(2*standard_deviation)

streamstats lets you gather an aggregate but represent it as a field per event. So each event gets an 'average' field that is the rolling average to that point. (A moving average, basically) This is great for comparing event values to aggregates, which is what you want to do here.

Alternatively, you can use eventstats, which does the same thing in giving each event a field with an aggregate value, but the difference here is it gathers the aggregate for the entire result set.

Play around with them and see what they do.

melonman
Motivator

Hi

A bit messy, but is this what you are looking for?

sourcetype=your_sourcetype [search sourcetype=your_sourcetype | stats stdev(reponse_time) as stdev | eval search="response_time>".(stdev*2) | fields + search] | stats count

melonman
Motivator

That "." is the String addition.
so the search field in subsearch specified in [] will look something like:

response_time>1234.56

and passed the response_time>1234.56 to the main search to fileter the events.

0 Karma

richnavis
Contributor

Works perfectly.. Thanks! BTW.. what does the"." character do in the subsearch?

0 Karma
Get Updates on the Splunk Community!

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...

Splunk APM: New Product Features + Community Office Hours Recap!

Howdy Splunk Community! Over the past few months, we’ve had a lot going on in the world of Splunk Application ...

Index This | Forward, I’m heavy; backward, I’m not. What am I?

April 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...