Why am I seeing "'SearchOperator:loadjob': Cannot find artifacts within the search time range" error in my saved search?

[EdgarAllenProse]

I've created a lot of scheduled saved searches to run daily at midnight, and ran into an issue with doing load job on my dashboards.

The Query In Question:

index=wineventlog  EventCode=4740 | stats count

• The dashboard is a Single Value visualization
• The time range I chose when creating the saved search was

=

Earliest/Latest=-14d@d | -7d@d

-
The savedsearch (I've taken out the event name and username) is called on the Dashboard with:

| loadjob savedsearch="user:search:daily_monitoring_prior_7_lockout"

The confusion for me is that only loadjobs for timeranges -14d@d and -7d@d show this error.
Identical queries in savedsearches containing -7d@d and now, run the loadjob fine.

I've tried changing the panel timepicker to follow the range in the search as well as changing it to last 24 hours and other variations, but all loadjobs with the problematic time range are running into issues. The saved search works, loadjob does not.

The related answers I found didn't work.

Any idea why I am getting the error regarding the search range?

[kbrown9392]

This is still a problem in version 7.2.3.

[gunzola]

Observed this issue with Splunk 6.6.3.
We saw, that a dashboard using scheduled reports for panels kept on running the searches and not using the last result from schedule.
Problem is related to the frequency of the scheduled report. Seems like you need at least 2 completed scheduled searches for the dashboard to show the pre-searched results.
The problem can be identified simply by running search:
| loadjob savedsearch="::
Which then would fail to show results of last scheduled search run.
We use Splunk for some monthly reports, where there usually always are only one result of the scheduled search available (present in dispatch-dir). Workaound is to have the schedule run the report twice within the month. Eg. 01:00am and 02:00am on the first date of month.
We do not see the issue in Splunk version 7.0.0
The problem has been reported to Splunk (support-case #531503).

[EdgarAllenProse]

So the problem solved itself, I wish I could give a detailed answer for what fixed it.

Basically the scheduled search has run twice and now I do not get the errors.

Maybe this is something related to custom searches with saved search?

[yannK]

My bet is that the load job only works once at least one job was already executed.
So you had to wait for the next daily schedule for it to work populate a job and have it available.

[yannK]

I assume that you have a scheduled job running the jobs weekly.
By default, splunk will keep in the dispatch folder the search artifacts for the last 2 runs of a scheduled search.
(based on the internal)

It may be why you still have the current week, the previous week, but not the last 2 weeks.
-> check in savedsearches,conf you may be able to specify a different retention for your dispatch artifact on a per search basis.

OR maybe the fact that the timerange is over 2 weeks fails because it requires to merge 2 jobs to answer the question, and it does not.

Have you tried to use report acceleration instead of loading jogs ?

[EdgarAllenProse]

I'm sorry I wasn't all that clear with that part.

I have 2 SingleValue visualization panels:

• One panel for last 7 days (-7d@d to now)
• One panel for the prior 7 days (-14d@d to -7d@d)
• both panel's timeframes were added in the creation of the knowledge object.

I am not trying to merge them or get a 14 day look. The saved searches were created yesterday and they run every night at midnight and have 24 hour retention. There hasn't been 2 job runs yet. Midnight was the first.

Another thing to note:

• if I click "open in search" on the broken panel, I get the error in search
• if I view recent jobs in settings -> searches, the logs are there, and it loads the last scheduled run