Splunk Search

Timechart & Span

paulf
Explorer

Hi,

I am collecting some disk performance stats via a Splunk Forwarder from a Windows Server.

I am now trying to graph the disk stats over the last 24 hours using the below.
sourcetype="Perfmon:LocalPhysicalDisk-SQLWEB" host=SQLWEB counter="Avg. Disk sec/Write" | timechart span=3s avg(Value)

I am using a 3s span because i am trying to show more accurate information.

Issue1
When using the 3s span, despite having last 24 hours set, it shows me only the last 20mins. This is really annoying as i am trying to compare the data to another tool, which is able to show me more accurate avergages over 24 hours. If i increase the span, the data becomes less and less accurate in terms of showing "spikes".
Is it possible to have the 3s, or any span, and have Splunk plot the data regardless of plot points, as it seems to be obbeying some kind of rule that dictacts span=x can only ever show x minutes.

Issue2
The data returned is for example 0.013, which is 13ms, is it possible to somehow * 1000, to plot 13, instead of 0.013?

Thanks

Paul

Tags (1)
0 Karma
1 Solution

melting
Splunk Employee
Splunk Employee

Issue 1: jscharts like Ayn said has limitations both around browser performance as well as pixel density. This page in our docs tries to explain more about this. There are a few ways to customize this, but I would try using a smaller timerange if you want the small spans.

Issue 2: I answered a similar question here:
Essentially you can use eval in the timechart cmd. ie:
... | timechart span=3s eval( 1000 * avg(Value)) as field

View solution in original post

0 Karma

melting
Splunk Employee
Splunk Employee

Issue 1: jscharts like Ayn said has limitations both around browser performance as well as pixel density. This page in our docs tries to explain more about this. There are a few ways to customize this, but I would try using a smaller timerange if you want the small spans.

Issue 2: I answered a similar question here:
Essentially you can use eval in the timechart cmd. ie:
... | timechart span=3s eval( 1000 * avg(Value)) as field

0 Karma

Ayn
Legend

It's not a limitation with timechart you're seeing, but rather a limitation with the chart. What Splunk version are you on? I know there was a limit of about 10000 data points or so with previous Flash charts, but I don't remember if it's still a problem with the JSChart module that is used by default by newer versions of Splunk.

Regardless, a 3 second resolution to a chart covering 24 hours, that's 28800 data points. Even at 1 pixel per data point, that's pretty insane. What kind of resolution are you using on your monitor that you would require that kind of resolution in your graph? If it's spikes you're looking for, perhaps you should be using max() rather than avg()?

paulf
Explorer

hi, i am running 4.3.4.

fair point about the max, im wondering now if the other tool is actually using max, because it says the max was 825.4, and looking at the plotted data on the graph, it shows 825.4 rather than an average...

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...