Looks good. I'll check out the monitor app.
In the short term I've come up with:
| metadata type=hosts | search host=#HostICareAbout# OR host=#HostICareAbout# | eval mytime=strftime (recentTime, "%y-%m-%d %H:%M:%S") | eval currentTime=strftime(now(), "%y-%m-%d %H:%M:%S") | eval minutesAgo=round(((now()-recentTime)/60),0) | table host,lastTime,recentTime,mytime,currentTime,minutesAgo | where (abs(minutesAgo) < 60)
This query will return a list of hosts that I care about which haven't sent any events within the last 60 minutes (the abs if for detecting when we have hosts in other TimeZones not properly configured). We're thinking of adding a local splunk metric file to be monitored so in case a system just doesn't have anything to forward it'll still forward an entry.
We're trying to figure out a simple file to monitor that won't impact our indexing volume.
... View more