I'm trying to find outlier using IQR method suggested by Splunk. I wonder why the statistics only shows 10,000 results !
https://docs.splunk.com/Documentation/Splunk/6.5.0/Search/Findingandremovingoutliers
are you using the sort
command? sort automatically truncates results. try |sort 0 -<fieldname>
http://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/Sort
can you share your search query?
| eval _time=(round(strptime(time, "%Y-%m-%d %H:%M:%SZ")))
| eventstats median("Run_Time") as median p25("Run_Time") as p25 p75("Run_Time") as p75
| eval IQR=(p75-p25)
| eval lowerBound=(median-IQR)
| eval upperBound=(median+IQR*20)
| eval isOutlier=if('Run_Time' < lowerBound OR 'Run_Time' > upperBound, 1, 0)
| fields "_time", "symbol", "sourcetype", "time", "Run_Time", "lowerBound", "upperBound", "isOutlier"
| table _time, symbol, sourcetype, time, Run_Time, lowerBound, upperBound, isOutlier
| sort -isOutlier
are you using the sort
command? sort automatically truncates results. try |sort 0 -<fieldname>
http://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/Sort
Adding |sort 0 - helped. But in statistics tab it still shows as 10,000 entries.
Is this something to do with IQR approach ?
While I attempted the "lower and upper boundaries of an acceptable range to identify outliers" , there also it was 10,000 values !!!
can you share your search?
This is using IQR.
| eval _time=(round(strptime(time, "%Y-%m-%d %H:%M:%SZ")))
| eventstats median("Run_Time") as median p25("Run_Time") as p25 p75("Run_Time") as p75
| eval IQR=(p75-p25)
| eval lowerBound=(median-IQR)
| eval upperBound=(median+IQR*20)
| eval isOutlier=if('Run_Time' < lowerBound OR 'Run_Time' > upperBound, 1, 0)
| fields "_time", "symbol", "sourcetype", "time", "Run_Time", "lowerBound", "upperBound", "isOutlier"
| table _time, symbol, sourcetype, time, Run_Time, lowerBound, upperBound, isOutlier
| sort -isOutlier
you're going to need | sort 0 -isOutlier
on the end to remove the 10,000 limit. the rest of it shouldn't matter. you also don't need both fields
and table
, you can just use one or the other.
Thank you @cmerriman