I want to run a search in splunk to find out that all the devices attached to the splunk server are generating logs. If I dont hear from a device I should receive a alert.
thanks
Here's a quick way to check for hosts not having sent data in a while:
| tstats latest(_time) as latest where index=* by host | where latest < relative_time(now(), "-1d")
Change the -1d
according to your needs, run the above search over at least twice the scheduled interval.
We had similar siutation and a heartbeat from the other system comes once in every 5 minute. So I'm doing that as a "saved Search" and run every 1 minute and checks data for the previous 6 minutes to see if there is a heartbeat. Else alert.
The logic is:
index=myindex earliest=-6m@s <your_some_more_Search> | stats count as COUNT_HEARTBEAT | where COUNT_HEARTBEAT=0
Hi ashari,
check out the Splunk Deployment Monitor there you can find some nice saved searches related to this topic ... keyword: missing forwarders
hope that helps ...
cheers, MuS
sure, although there are a lot of search macros in the saved searches. If you look at the searches you will get an idea how it can be done.
Otherwise use a lookup file which contains all your forwarder hostnames and check if all of them were seen in the past x minutes. Take a look at this http://answers.splunk.com/answers/73268/search-for-hosts-in-a-lookup-but-not-in-splunk
can we do this without splunk deployment monitor app.