Splunk Search

How to add the average of the top 5 percent of values, but remove the max value, to my timechart?

jlamb3
New Member

What I have:

"Properties.MetricType"=ResponseTiming AND "Properties.Http_Request_Path"=/BackflushInputs | timechart max("Properties.ElapsedMilliseconds") avg("Properties.ElapsedMilliseconds")

This gives me a very nice graph: alt text

Because the Max value can be a significant outlier from than the rest, I'd also like to display the average of the top 5% values, minus the max value.

I've found answers to somewhat similar questions here:
https://answers.splunk.com/answers/75965/top-percentage-out-of-total-events.html
https://answers.splunk.com/answers/61711/average-time-on-only-top-results.html
...but I'm having trouble morphing them to exactly what I want. Any ideas on how I can achieve this, adding it to my current chart? Thanks!

Tags (3)
0 Karma

gokadroid
Motivator

something like:

your query to return events
| eventstats perc95(Properties.ElapsedMilliseconds) as interestedValue
| search Properties.ElapsedMilliseconds < interestedValue
| stats avg(Properties.ElapsedMilliseconds) as Avg
0 Karma

somesoni2
Revered Legend

You may want to explore function percX aggregation function available in timechart (and stats/chart). This gives the X percentile value, so if you use perc95("Properties.ElapsedMilliseconds") in your timechart, it should skip the top 5% outliers.

http://docs.splunk.com/Documentation/Splunk/6.5.1/SearchReference/CommonStatsFunctions#Aggregate_fun...

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...