Splunk Search

How to display 86400 points on timechart?

brandy81
Path Finder

Hi, I need your helps. I am trying to display 86400 points with timechart. I did applied configuration below. The version is 7.2.6.

visualizations.conf
data_sources.primary.params.count = 0

web.conf
jschart_truncation_limit = 0
jschart_truncation_limit.chrome = 0
jschart_series_limit = 0
jschart_results_limit = 0

test.xml
0
0​

SPL : index=_internal earliest=-d@d latest=@d sourcetype=splunkd | timechart span=1s count

But, the chart display only 50000 points....(It displays only from 00:00AM to around 2:00PM.) Is it actual limitation? How to display 86400 points all together?

0 Karma

tscroggins
Influencer

(Edit: Removed note about limit option. It's only applicable to split-by groups.)

If your goal is to display 86400 points, you'll need a browser viewport at least 86400 pixels wide plus chart overhead. On smaller displays, the chart will show a subset of evenly distributed values across the timeline. Depending on your desired chart width, you may want to optimize how data is aggregated in the timechart command rather than letting the chart drawing code handle it.

0 Karma

woodcock
Esteemed Legend

The limit is 50K and I have never tried to extend it. What you can do is be far more creative about the visualizations and options that you use. For example, it is often possible to use the trellis feature to break out one of the fields so that each value of this field has a different panel. Another option is to split the Y-legend into several panels such as 3 with A*-F*, G*-R*, and T*-Z* or similar.

0 Karma

niketn
Legend

@brandy81 if you are actually querying metadata fields you can refer to the following answer by @somesoni2: https://answers.splunk.com/answers/527089/different-results-for-error-specified-span-would-r.html

| tstats count where index=_internal earliest=-1d@d latest=@d by _time span=1s

If not you can try to bucket time using bin command as 1s and use chart command with by _time instead of timechart command.

index=_internal earliest=-1d@d latest=@d 
| bin _time span=1s
| chart count by _time

Ideally if you really know that users can search for such small buckets and each search is going to give the user with high volume data, you can change the maxresultrows in limits.conf as per the following answer: https://answers.splunk.com/answers/64666/the-specified-span-would-result-in-too-many-50000-rows.html

For overcoming Chart with 10K limit in charting, as @Sukisen1981 mentioned you can refer to one of my older answers to set the same through Simple XML configuration as well. PS: For Single Value the SimpleXML configuration is available from Splunk version 7 and higher.: https://answers.splunk.com/answers/658129/single-value-viz-these-results-may-be-truncated-th.html

Splunk Docs link for Chart Display Issues with Data Truncation

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

brandy81
Path Finder

HI, @niketnilay Thank you for your comment. Unfortunately, it doesn't work. I set Simple XML option, of course, and changed all relevant configurations on visualizations.conf, limit.conf, web.conf. I did test on 3 different versions and tried to figure out what's wrong. It's weird... Is it really working?

0 Karma

Sukisen1981
Champion

hi @brandy81
This might sound like a stupid question but I believe we are in a situation, where we just have to do hit n trial.
so, here is how my dashobard xml looks like

<dashboard>
  <label>trunc</label>
  <row>
    <panel>
      <chart>
        <search>
          <query>index="_audit" 
| timechart span=1sec count</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
        </search>
        <option name="charting.chart">column</option>
        <option name="charting.chart.resultTruncationLimit">90000</option>
        <option name="charting.data.count">0</option>
        <option name="charting.drilldown">none</option>
      </chart>
    </panel>
  </row>
</dashboard>

2- my web.conf file

jschart_truncation_limit = 90000
jschart_truncation_limit.chrome = 90000
jschart_truncation_limit.firefox = 90000
jschart_truncation_limit.safari = 90000
jschart_truncation_limit.ie11 = 90000
jschart_series_limit = 100

3- what splunk version are you on? when i ran this on 6.3 , it failed to have any impact but it should work on 7.x
4- this is really silly, but did you re-start splunk after making the changes, and clear you browser cache?

0 Karma

brandy81
Path Finder

Hi, @Sukisen1981 Thank you so much.
1-2. I changed configurations as you suggested.
3. I try 7.2.6 & 7.3.1. Both fail.
4. Of course, I restarted and cleared browser cache...
What's wrong... 😞

0 Karma

Sukisen1981
Champion

hi @brandy81
I am really sorry, but I have reached the end of my knowledge 😞
It is perhaps time to raise a support case with Splunk or wait for some other member who has more knowledge on this than me.
Once again, sorry that I could not be of much help

0 Karma

brandy81
Path Finder

Hi, @Sukisen1981 Now I've resolved it by myself. I was due to a maxresult value of restapi on limits.conf. Since I've changed it to 200000 (0 is not working), it works...

[restapi]
maxresultrows = 200000

Hope it would help others who have same problems. Thank you for your willingness to help me. Thanks again.

0 Karma

Sukisen1981
Champion

Hi @brandy81
Finally 🙂
I think this was a long haul.
I definitely think you should answer this question yourself and accept the answer, as it is that good and I am sure will be of help in the future.
Now, I have 2 questions:
1- By just making the change in limits.conf AND not editing the xml or web/visualize.conf does it work?
2-I was testing on the delivered _audit index, which has no relation to any api (it is merely the audit record in splunk , in DB manner of speaking) , so how does THAT work? I am sure if you try my query on the _audit index you will still receive the limitation error?
Lastly, I think you owe yourself a cup of coffee /hot chocolate/beer depending on what you prefer, I can understand how much time and effort you spent on this, so post an answer to the question and let me upvote the answer 🙂 : )

0 Karma

brandy81
Path Finder

Hi, @Sukisen1981 Yes.. It was a long haul but it's worth. The answers for your questions are here:
1. web.conf + limits.conf + xml must be configured. These are necessary combination.
2. Of course I tested with _internal/_audit and my own logs. The reason it works with internal logs is because when the browser get data from splunk, it calls restapi. From web browser side, it doesn't matter if the logs are internals or externals.

You must know how much I am happy! 🙂 Thank you for your help. Hope it helps.

0 Karma

brandy81
Path Finder

test.xml
< option name="charting.data.count" > 0 < /option >
< option name="charting.chart.resultTruncationLimit" > 0 < /option >

0 Karma

Sukisen1981
Champion

hi @brandy81 , yes there is a limit
see here -
https://docs.splunk.com/Documentation/Splunk/7.3.1/Viz/ChartDisplayissues
this also describes settings to over ride the limitations
how you chart looks like after changing the config is another matter 🙂

brandy81
Path Finder

@Sukisen1981 Thank for you the answer. Of course I saw the link. All configuration I did were from the link. The point is it didn't work. I set all options to "0", means "no limit". But only limited points are displayed. Could you please anybody let me know how to do it?

0 Karma

Sukisen1981
Champion

can you test by changing this
< option name="charting.chart.resultTruncationLimit" > 5000 < /option >
if this works you can increase this value to 90k and test
dont think you need < option name="charting.data.count" > 0 < /option >
how many series (lines) do you have in the timechart? it should handle upto 100 by default.
last resort is this - jschart_truncation_limit setting in $SPLUNK_HOME$/etc/system/local/web.conf
but of course this is not recommended as this will apply to all charts running on your instance

0 Karma

Sukisen1981
Champion

hi @brandy81 any luck?

0 Karma

brandy81
Path Finder

Hi, @Sukisen1981 Thank you for yours answer. It works only when the points are less than 50000. I think 50000 points are limits in timechart display in Splunk. I need 86400
(1 day with 1 second unit) points displayed on timechart. It seems impossible...

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...