I have a dashboard with 2 saved searches loaded to it, when I try to load the dashboard it takes around 1-2 minutes for the graphs to render (only 1 is able to render each time - I have no idea why) and the CPU/MEM on the splunk machine spikes.
Here are the two saved searches:
This should show the avg of mem usage for the process 'splunkd' on two hosts
index="os" sourcetype="ps" host=stage.site OR host=testbuild.site | multikv fields pctMEM, COMMAND | where COMMAND="splunkd" | append [stats sum(pctMEM) as pctMEM by _time,COMMAND | eval host=split("stage.site;testbuild.site",";")
| mvexpand host ] | timechart avg(pctMEM) by host | eval count=count-1
This should show the avg of cpu usage for the process 'splunkd' on two hosts
index="os" sourcetype="ps" host=stage.site OR host=testbuild.site | multikv fields pctCPU, COMMAND | where COMMAND="splunkd" | append [stats sum(pctCPU) as pctCPU by _time,COMMAND | eval host=split("stage.site;testbuild.site",";")
| mvexpand host ] | timechart avg(pctCPU) by host | eval count=count-1
The time range for these two searches is defined to -24h@h.
I have another dashboard with the exact same saved searches but with a different time range of -1h@h and that dashboard loads correctly without spiking the mem/cpu usage and it does not hang.
This is how I load the 2 saved searches in my dashboard:
<module name="HiddenSavedSearch" layoutPanel="panel_row2_col1" group="splunkd CPU Usage last 24h" autoRun="True">
<param name="savedSearch">splunkd CPU Usage last 24h</param>
<param name="groupLabel">splunkd CPU Usage last hour (24h)</param>
<module name="HiddenChartFormatter">
<param name="charting.chart.stackMode">default</param>
<param name="charting.secondaryAxisTitle.text">% CPU</param>
<param name="charting.legend.placement">left</param>
<param name="charting.chart">line</param>
<param name="charting.chart.nullValueMode">connect</param>
<module name="JSChart">
<param name="width">100%</param>
<param name="height">280px</param>
<module name="ConvertToDrilldownSearch">
<module name="ViewRedirector">
<param name="viewTarget">flashtimeline</param>
</module>
</module>
</module>
<module name="ViewRedirectorLink">
<param name="viewTarget">flashtimeline</param>
</module>
</module>
</module>
<module name="HiddenSavedSearch" layoutPanel="panel_row2_col2" group="splunkd MEM Usage last 24h" autoRun="True">
<param name="savedSearch">splunkd MEM Usage last 24h</param>
<param name="groupLabel">splunkd MEM Usage last 24h</param>
<module name="HiddenChartFormatter">
<param name="charting.chart.stackMode">default</param>
<param name="charting.secondaryAxisTitle.text">MEM %</param>
<param name="charting.legend.placement">left</param>
<param name="charting.chart">line</param>
<param name="charting.chart.nullValueMode">connect</param>
<module name="JSChart">
<param name="width">100%</param>
<param name="height">280px</param>
<module name="ConvertToDrilldownSearch">
<module name="ViewRedirector">
<param name="viewTarget">flashtimeline</param>
</module>
</module>
</module>
<module name="ViewRedirectorLink">
<param name="viewTarget">flashtimeline</param>
</module>
</module>
</module>
... View more