Getting Data In

Search for results NOT found in the last 24hours?

ericsales
New Member

Edit: rephrasing the question a bit

I have a job that is remotely triggered which should be run at least once within a 24 hour period. The start message (i.e. "Job Triggered") appears in /var/log/messages. What is the optimal way to search/report for hosts that DO NOT have the Job Triggered message within a 24 hour period?

So far, I have this in the search cmd:
source="/var/log/messages" host="*" "Job Triggered." earliest=-1d | dedup host | stats count by host

This shows the results, but doesn't tell me how many hosts didn't have the Job Triggered in that period.

Tags (1)
0 Karma

sowings
Splunk Employee
Splunk Employee

In order to evaluate against history (to find a gap), you'll have to collect some history. A way that this is achieved in the Deployment Monitor app (which ships with Splunk) is to utilize a summary index that's used to "remember what is seen". Another way would be to use | inputlookup combined with | outputlookup to create a CSV file that has some history.

Ultimately, you'd end up with a list of "hosts we've seen kick off the job over all time, and the last time they ran it", and then perform some time math against | eval this_time=now() to see if it's > 24h.

Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

REGISTER NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If ...

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...