Dashboards & Visualizations

Is it possible to schedule a dashboard to run panel searches at 12:00AM, so every time someone opens the dashboard, they display cached results?

ranjithfs1
Explorer

I have a dashboard with close to 20 panels. (Each search is taking anywhere between 1 to 5 min to run) The dashboard doesn't have any form elements. For all the panels, the time range is one of the following

  • Earliest: -7d@d Latest: @d
  • Earliest: -2d@d Latest: @d

I would like the dashboard to run only once every day at 00:00, and every time someone opens the dashboard display the cached results.

Is it possible to do this by changing a single setting or something like that?

I couldn't find any option like that. So, I have converted each of the panels to a report and enabled scheduling on each of the reports to run once everyday at 00:00. (<u>with no email or alert actions</u>)

<u>(I have configured the scheduling today itself. It is possible that the scheduled search never happened till now. I'm not sure of the timezone of the Splunk server)</u>

Half of the panels are displaying the following error.

In handler savedsearch: Search not executed: The maximum number of historical concurrent system-wide searches has been reached. current= 15 maximum= 15 Searchld=<MY_USERID>

Earlier, when the dashboard was made using inline searches, 15 of the panels would keep loading and the status for the remaining used to be Queued. Eventually, all the panels used to load without any errors. Why is the dashboard with embedded reports not behaving in the same way?

Will I encounter the same problem when all the 20 scheduled searches get dispatched at 00:00?

Also, the panels are executing the searches every time. They don't seem to be using cached results. I am sure about this as the load time of the results is in minutes. Why is this so?

Splunk version: version=6.2.6

0 Karma

_jgpm_
Communicator

The way I speed up my dashboard results is by creating an intermediate csv.

[ Long tedious search ] --> | outputcsv  csvname.csv --> Dashboard source "... <search> <query> | inputcsv csvname.csv ..."

Loading a csv is near instantaneous.

jboucher_splunk
Splunk Employee
Splunk Employee

Dashboard panels don't really cache information. They run each panels search at the time of the dashboard loading. However, you can schedule a report and import the results of the scheduled report into a dashboard panel. For instance, if you scheduled a report to run once a day at 00:00 then the dashboard would show the results of the scheduled report.

It seems like you are trying to do that now, but you get an error on the number of historical searches. You can fix this error by updating the limits.conf file in $SPLUNK_HOME/etc/system/local/. Which would allow for more historical searches. There is another Splunk answers topic that explains this at: https://answers.splunk.com/answers/54674/how-to-increase-the-maximum-number-of-concurrent-historical...

However, I personally think that the real issue is how long it takes to conduct a search for a week's worth of data. I would definitely pair down my dashboard to 10 or less panels, then create a second dashboard with the remaining panels. Or just link to the reports in the dashboard for information on the dashboard that is not as critical. You could also post some of your searches here for the community to review if you would like them to help you optimize the search query. This is a much better option than modifying your limits.conf file, since that could have some negative results in your system's performance. With 15 concurrent searches, the server sounds like it might be a little underpowered anyway, and optimizing your searches should make it present your results faster with less wear and tear on your system.

Here is a link to a doc that talks about search optimization: http://docs.splunk.com/Documentation/Splunk/6.2.5/Search/Writebettersearches

Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...