OPSEC logs stopped, LEA script set to run and works by hand?

[gowen]

Our Splunk server is configured to pull firewall logs from the Check Point using OPSEC/LEA (specifically, the lea-loggrabber-splunk tool that comes with Splunk).

The script is configured to run every 60 seconds and is Enabled.

If I log in at the command line and run lea_loggrabber by hand, it is successfully connecting to the Check Point and pulling log entries.

However, if I search for entries of this sourcetype, it goes all the way back to 11/24 before finding any. And now that I think about it, I haven't seen email alerts related to a search on this data for about that length of time.

So, my firewall logs aren't making it into Splunk. How do I debug this problem? Can I find logs of Splunk invoking the lea_loggrabber tool somewhere? Is it possible there's a reason they aren't getting imported? We exceeded our license somewhere close to the timeframe this stopped in, and used a -reset license to cover while we're purchasing more licenses, but is it possible that the brief period of license overage caused Splunk to stop indexing this data input?

Any help is appreciated.

[gowen]

A disable/re-enable of the script through the Splunk Manager interface was sufficient to restart this indexing; it grabbed all of today's logs after the dis/re and has continued to update every minute since then.

I will need to see if I can run the script against rotated Check Point logs; IIRC the command line supports it and I just need to figure out how to run it and get it shoved into the indexer.

Any help on debug steps would still be helpful for next time. Where does Splunk log each time it runs the script? (e.g., can I see if the script was running but not indexing, or just not running?)