How can I force my dashboard to continously build without restarting every time any user opens it?

[bowmanj1]

So here is the situation That I am in. At any particular time we are only working on one set of data. This data gets purged every month and we start over from scratch with new data. I want to create a dashboard that gives overall information over this data. the problem that i am having is the dashboard keeps restarting the search everytime it is opened. This means as we ingest more data and navigate back to the dashboard instead of adding the results to the dashboard it starts over and re-performs the search on the old and new data. With the amount of data we collect it would take several days for the report to finish and then if you refresh the page or get disconnect it has to start over.

I would like a dashboard that as data is ingested it continuously updates itself. Once we have ingested the full data set and the dashboard has finished, any user should be able to go to the dashboard and see the most current information. Since several of us will navigate to the dashboard there is no need for it to start a new job everytime just have one completed job.

Example: Splunk is empty, brand new. we start capturing pcap data and splunk is ingesting it over a 1 week period. during that one week period i create a dashboard looking for a count of how many packets are going to port 22. as we ingest data, day to day, i can navigate to the dashboard and see our current count of packets going to port 22. then by the end of the week when i navigate back i see the overall count. (how can this be done without the dashboard restarting the search every time i go to it?)

[adonio]

have you tried to power the dashboard by scheduled reports?
have you considered summery indexes?
report acceleration?
what about some base search to relief the pain of running the same basic search for couple panels?
did you learn about the loadjob or savedsearch commands
with that being said,
what are the dashboard's requirements? how many panels do you have? how many form inputs?
how many users supposed to look at the dashboard at any given moment?
for your example use case with packets, i will recommend summary index,
capture every day the total count of packet -> now you have 1 event, after a week you have 7 events
run a search on the summary index and it will return in a fraction of a second.

hope it helps

[bowmanj1]

The plan is to have several different dashboards that are constantly updating as we ingest pcap data. we are looking at 3 different users who will pull up the dashboard at random times. But when they do they see the most current up to date dashboard. I am trying to avoid the dashboard having to restart the scan everytime one of them opens it.

So here is one example. We are trying to find ips on a network associated with port 22 traffic. As we index pcap data into splunk. here is a basic search

" sourcetype="pcap:csv" dst_port=22 OR src_port=22 | stats count by src_ip"

This search will list Ips associated with port 22 traffic. so during the day these 3 users will need to refer back to this list to see new ips that pop up. The current problem i am having is every time a user opens this dashboard it starts a new search job. So if i have 3 users that is 3 jobs that restart every time they open the dashboard. Ideally it would be 1 job constantly updating. This would give all 3 users the ability to open the dashboard and see the most current information, then by the end of the week when we are done ingesting pcap data we would have 1 full complete report.

[adonio]

i see,
please refer to one (or more) of the methods i suggested above as each will increase your effectiveness in its own way.
from the sourcetype name, it seems like your data is in csv format, if you are using the prebuilt csv configurations for indedxed extractions in splunk, you can leverage tstats and have your queries run 100 times (or more) faster. example search:
| tstats count as _events_count where index=* sourcetype = "pcap:csv" dst_port=22 OR src_port=22 by src_ip
give it a shot

[ddrillic]

Report acceleration seems to be a good choice here...