How to search and display the average time disk transfers took to complete in seconds?

[idab]

Hi ,

Is there an easier way to write a search to separate and display stats values within a 1min interval/bucket for Avg. Disk sec/Transfer, latency and, disk drive(instance) on separate columns?

I tried with this search, but no luck:

index=perfmon  | search counter="Avg. Disk sec/Transfer"   | dedup host | Host="*" | eval latency =  round(latency,3) | eval percent= round(percent ,3) | top  limit=10 Host    _time instance latency source

[lguinn2 · Accepted solution]

I don't know what you mean by a "one minute interval/bucket", but I can definitely recommend some improvements to your search

index=perfmon counter="Avg. Disk sec/Transfer"  Host="*" 

should be your base search. Don't break it up; that is much less efficient. Second, there is no field called "latency" in the data that you are showing. How do you calculate latency? Where does that come from?

Finally, what is the actual result that you want to see? Do you want to see the most recent value for some statistics? Would you like to plot some values over the last 1 minute? Do you want to update this continuously? Is it going to be part of a dashboard, or a report that is run on demand...

[lguinn2]

index=perfmon counter="Avg. Disk sec/Transfer"  Host="*" collection=PhysicalDisk earliest=-1m
| timechart span=1s first(latency) by Host

The above will give you a chart that has one line per host, showing at most one latency value per second over the past minute. Do the same thing, substituting the field for "avg disk transfer value" (probably the Value field). Put both reports on a dashboard.

If you really, really want to have this on a single report, it will be more harder to read and understand. But here it is, I think. The following assumes that each event has both a latency field and a field named Value that contains the "avg disk sec/transfer"

index=perfmon counter="Avg. Disk sec/Transfer"  Host="*" collection=PhysicalDisk earliest=-1m
| eval dataValue="latency:" + tostring(round(latency,3)) + "," + "avg disk sec/transfer:" + tostring(round(Value,3))
| makemv delim="," allowempty=true dataValue
| mvexpand dataValue
| eval part=split(dataValue,":")
| eval category = Host + ":" + mvindex(part,0)
| eval dataPoint = tonumber(mvindex(part,1))
| timechart span=1s first(dataPoint) by category

I don't think you should mix logical and physical disk counters; I believe that they are different things. If you want the dashboard to refresh every minute, you can edit the dashboard XML and add the following option to the panel:

<option name="refresh.auto.interval">60</option>

[idab]

Hi Iguinn,

Thanks for the feedback. This was nicely done.So, I have another question:

How can I get the top 10 host to display the on the dashboard?
I tried to use the top command to limit the search to 10, but no joy.

index=perfmon counter="Avg. Disk sec/Transfer"  Host="*" collection=PhysicalDisk earliest=-1m
 | eval dataValue="latency:" + tostring(round(latency,3)) + "," + "avg disk sec/transfer:" + tostring(round(Value,3))
 | makemv delim="," allowempty=true dataValue
 | mvexpand dataValue
 | eval part=split(dataValue,":")
 | eval category = Host + ":" + mvindex(part,0)
 | eval dataPoint = tonumber(mvindex(part,1))
 | timechart span=1s first(dataPoint) by category | top limit=10 host.

[piebob]

if this is an entirely different question, ask it as a separate question on the site, please

[idab]

It's not actually.Just need further clarification 🙂

[lguinn2]

Try this - note that you need to identify the top 10 hosts, but still split by both the host and the data value. Therefore, identifying the top ten hosts must come early in the process. You didn't say how to define the top 10, so I decided to show the 10 hosts with the highest latency. I used a subsearch for this, because the computation for identifying the hosts with the highest latency is quite different than the main search.

index=perfmon counter="Avg. Disk sec/Transfer"  Host="*" collection=PhysicalDisk earliest=-1m
   [search index=perfmon counter="Avg. Disk sec/Transfer"  Host="*" 
                 collection=PhysicalDisk earliest=-1m
      | stats max(latency) as latency by host
      | sort 10 -latency
      | fields host ]
| eval dataValue="latency:" + tostring(round(latency,3)) + "," + "avg disk sec/transfer:" + tostring(round(Value,3))
| makemv delim="," allowempty=true dataValue
| mvexpand dataValue
| eval part=split(dataValue,":")
| eval category = Host + ":" + mvindex(part,0)
| eval dataPoint = tonumber(mvindex(part,1))
| timechart span=1s latest(dataPoint) by category

[idab]

Hi Iguinn,

So , I was going over the last response you put in for :

index=perfmon counter="Avg. Disk sec/Transfer"  Host="*" collection=PhysicalDisk earliest=-1m
 | timechart span=1s first(latency) by Host

So, I made a change to round-up the values to 3 decimal places but no joy:

index=* counter="Avg. Disk sec/Transfer"  Host="*" collection=PhysicalDisk earliest=-1m    | timechart  span=5m first(Value) AS rawData by Host   | eval rawData = round(rawData,2)

Tried to round-up the final value but no changes . How do I write the SPL to show the round-up values to 3 decimal places?
/storage/temp/59180-roundup.jpg

[idab]

Hi Iguinn,
Pls ignore the last post - I was able to figure it out.Thanks

[idab]

Hi Iguinn,

So, I have a field on my log called latency - my bad ; I didn't include it in my earlier snap-shot.
But it does exist.See attachment.

/storage/temp/56231-latency.jpg

Finally, what is the actual result that you want to see? I want to see a graph representation how the the avg. disk transfer value , latency , _time stamp by host using the bar char

Do you want to see the most recent value for some statistics? Yes

Would you like to plot some values over the last 1 minute? Yes

Do you want to update this continuously? When you say update do you mean - run the search over different time period /span?

Is it going to be part of a dashboard, or a report that is run on demand... yes as a report and eventually added to a dashboard.

Thanks for the feedback.