Splunk Search

My search does not work if it is scheduled to run for generating lookup table

tonopahtaos
Path Finder

Hi,

I have following lookup cron job defined in savedsearches.conf (the search condition is simplified for this discussion):

[AD Password Change by Domain]

cron_schedule = */15 * * * *

enableSched = 1

dispatch.earliest_time = -3d

dispatch.latest_time = now

run_on_startup = true

dispatch.lookups = 0

description = Create AD password change statistics lookup file.

search = EventCode=4724 | eval time=strftime(_time, "%c") | stats dc(time) as password_changes by domain | outputlookup stat_ad_password_changes_by_domain

It is expected to run every 15 minutes and refresh the stat_ad_password_changes_by_domain.csv file. But every time it runs, stat_ad_password_changes_by_domain.csv file is set to size 0(there is no content in such file).

The search itself works. If I copy the search (EventCode=4724 | eval time=strftime(_time, "%c") | stats dc(time) as password_changes by domain | outputlookup stat_ad_password_changes_by_domain) and run it from Splunk console, it works and displays following result:

domain password_changes

MYDOMAIN.COM 1

In the mean time, stat_ad_password_changes_by_domain.csv file contains the right content. But after 15 minutes, the csv file size is changed to 0 since cron job run such search and refresh the csv file.

I do have multiple other good lookup tables. If I switch this search with a good lookup table, it breaks such good lookup table. So, this search really has problem when it is run from cron for generating lookup table although it works fine when running from Splunk console.

Anybody has any idea why this search has problem? Also in general, what is the way to debug such problem? Since running such search from Splunk console is working, it has to be related to cron job for generating lookup table. But i have no idea how to debug this.

Thanks in advance!

John

Tags (2)
0 Karma

Takajian
Builder

If the search running in splunk console is fine, did you check earliest time and latest time? If both time is same as you did in splunk web console, you will need to case open to splunk support.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...