Why are scheduled searches not running at their proper times with cron schedules in Splunk 6.1.1?

[ebastos]

Hello!

I'm running Splunk Enterprise 6.1.1 and a user reported that his scheduled jobs are not executed at their proper times.
I confirmed this information and here is an example.

But the job actually ran at 3AM:

Same user has a job scheduled to run at 10 past midnight and it executed at 2AM. Another job scheduled for 5 past midnight executed just fine.
I tried looking at the internal Splunk logs and tried to find any obvious errors with
bin/splunk cmd btool --debug savedsearches list
but no luck so far.

I would appreciate any advice on this matter, please.

Regards,

[woodcock]

Did you ever figure out what was happening here?

[somesoni2]

Got a doubt here. The cron on the screenshot is scheduled for "5 past midnight (oo:05 AM)" and executes at 3 AM (INCORRECT Scheduling) and the text below says a job scheduled for "10 past midnight (00:10 AM)" executed at 2 AM (INCORRECT Scheduling) and another job scheduled for "5 past midnight (oo:05 AM)" ran fine (CORRECT Schedule). Is that correct?

[ebastos]

Sorry, let me clarify:

The user has multiple jobs. All scheduled around midnight (between 00:05 and 00:20).
Some of them run exactly on schedule. Some run at 2AM and some run at 3AM.

And I don't mean it's scheduled for 00:05 and run at 03:05 (three hours off). I mean 3AM on the dot, which makes no sense for me.

[woodcock]

The problem is that the scheduled job runs AS A USER (the user that saved it). Each user has a Timezone setting inside his profile under My User Name -> Edit account -> Timezone. When you say "3AM", you are actually saying "3AM as interpreted by this user's Timezone setting", which in your case, is 3 hours different than you think it should be.

[ebastos]

Thanks. I just checked that and the user has the correct time zone. I also compared with another user which I know by fact that has a working job and they matched.

Also a problem with the time zone would cause a full 2 or 3 hours mismatch, but as you can see on the screenshots the job is scheduled for 5 minutes past midnight and actually ran at 3AM on the dot.

[woodcock]

Based on your pictures, I thought we were talking about a 3-hour (and 5 minutes) difference. Your pictures don't match your text.