Splunk Search

My search does not work if it is scheduled to run for generating lookup table

tonopahtaos
Path Finder

Hi,

I have following lookup cron job defined in savedsearches.conf (the search condition is simplified for this discussion):

[AD Password Change by Domain]

cron_schedule = */15 * * * *

enableSched = 1

dispatch.earliest_time = -3d

dispatch.latest_time = now

run_on_startup = true

dispatch.lookups = 0

description = Create AD password change statistics lookup file.

search = EventCode=4724 | eval time=strftime(_time, "%c") | stats dc(time) as password_changes by domain | outputlookup stat_ad_password_changes_by_domain

It is expected to run every 15 minutes and refresh the stat_ad_password_changes_by_domain.csv file. But every time it runs, stat_ad_password_changes_by_domain.csv file is set to size 0(there is no content in such file).

The search itself works. If I copy the search (EventCode=4724 | eval time=strftime(_time, "%c") | stats dc(time) as password_changes by domain | outputlookup stat_ad_password_changes_by_domain) and run it from Splunk console, it works and displays following result:

domain password_changes

MYDOMAIN.COM 1

In the mean time, stat_ad_password_changes_by_domain.csv file contains the right content. But after 15 minutes, the csv file size is changed to 0 since cron job run such search and refresh the csv file.

I do have multiple other good lookup tables. If I switch this search with a good lookup table, it breaks such good lookup table. So, this search really has problem when it is run from cron for generating lookup table although it works fine when running from Splunk console.

Anybody has any idea why this search has problem? Also in general, what is the way to debug such problem? Since running such search from Splunk console is working, it has to be related to cron job for generating lookup table. But i have no idea how to debug this.

Thanks in advance!

John

Tags (2)
0 Karma

Takajian
Builder

If the search running in splunk console is fine, did you check earliest time and latest time? If both time is same as you did in splunk web console, you will need to case open to splunk support.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...