I'm trying to write a report that will show me the 1 minute time spans from the last two days where a specific machine had a average CPU utilization (the field is called Value) of 75 or higher. This is what I have so far, but I'm having trouble finishing it. Thanks in advance for any help.
index="perfmon" host="mymachine" collection="CPU Load" counter="% Processor Time" earliest="-2d@d"
Try this
index="perfmon" host="mymachine" collection="CPU Load" counter="% Processor Time" earliest="-2d@d"
| bucket _time span=1m
| stats avg(Value) as avg_CPU by _time
| where avg_CPU > = 75
Just add Value >= 75 to limit which events splunk returns, then add the following.
I am not sure how you want your minutes formatted, but here is what I have used in the past.
eval DATETIME=strftime(_time, "%D %H:%M") | convert timeformat="%D %H:%M" mktime(DATETIME) AS DATETIME| fieldformat DATETIME=strftime(DATETIME,"%D %H:%M")
For more options or google strftime.[CommonEvalFunctions][http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/CommonEvalFunctions]
Incase you have two values per minute, I am using max() to take the highest value for that minute. You can use avg(), min() etc..
| chart max(Value) by DATETIME
Note that this will leave some time periods missing. If you want every minute listed, use | timechart span=1m max(value)
The whole thing looks like this
index="perfmon" host="mymachine" collection="CPU Load" counter="% Processor Time" earliest="-2d@d"
Value>=75
| eval DATETIME=strftime(_time, "%D %H:%M")
| convert timeformat="%D %H:%M" mktime(DATETIME) AS DATETIME
| fieldformat DATETIME=strftime(DATETIME,"%D %H:%M")
| chart max(Value) by DATETIME
The Datetime conversion trick I learned from the Answers system. Wish I could find the right post and give them credit here. It is very useful.
Is a way to round up the max(Value) to 2 decimal places?
For any numerical values try: eval roundedMax=round(max(Value),2). This is specified more completely in the eval documentation. http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/CommonEvalFunctions
Potentially handy also are eval variants 'ceiling' or 'floor' to FORCE the rounding up or down.
Try this
index="perfmon" host="mymachine" collection="CPU Load" counter="% Processor Time" earliest="-2d@d"
| bucket _time span=1m
| stats avg(Value) as avg_CPU by _time
| where avg_CPU > = 75
This is exactly what I was looking for and very succinct. Thanks a ton!
@Iguinn Thanks.
Can you suggest how to restrict this search to a particular host where "avg_CPU" >75% for more than 5 minutes.
i want to list down those such hosts.
also in my query, i am doing it on "% Idle Time" <25%, hope that is also the same.
Thanks in advance