Good afternoon,
Long question and I hope I can explain it well enough. I'm using a third party file access logging product for my file servers. The third party logging server has a console where I view the logs but I also use an HEC to forward the same logs to Splunk from the third party logging server. The sourcetype in Splunk is always "httpevent". So the "host" value in Splunk always equals [logcollector] and the sourcetype always equals httpevent. Each event from logcollector has a field of "computer" which identifies what host the event originated from before getting to logcollector.
I want to search through all the "httpevent" types every five minutes and alert me on any "comptuer" that hasn't sent an event in "X" seconds. Each threshold is different so I created a lookup table with host, computer, max_delay, and admin. host is always logcollector, computer is one of the 15 hosts logging to logcollector. I have a similar search for some database connections so I figured I could just modify that one slightly but it's not working. See working search below. The quotes around the less than sign is intentional as it interpreted that as html and truncated everything after the first less than sign:
|tstats max(_time) as latest where index=[my index] by host, source, sourcetype
|search [|inputlookup DB_Source.csv|fields source]
|lookup DB_Source.csv source OUTPUT max_delay, admin
|eval current_delay=now()-latest
|where max_delay"<"current_delay
|convert timeformat="%a %b %d, %Y %I:%M:%S %p" ctime(latest)
|rename host as Host, latest as Latest, source as Database current_delay as "Current Delay (seconds)", max_delay as "Max Delay (seconds)"
So I figured I could modify the search to be:
|tstats max(_time) as latest where index=[my index] by host,source, sourcetype
|search [|inputlookup LogCollectorHosts.csv|fields computer]
|lookup LogCollectorHosts.csv computer OUTPUT max_delay, admin
|eval current_delay=now()-latest
|where max_delay"<"current_delay
|convert timeformat="%a %b %d, %Y %I:%M:%S %p" ctime(latest)
|rename host as Host, latest as Latest, computer as Computer current_delay as "Current Delay (seconds)", max_delay as "Max Delay (seconds)"
But that search just produces zero results. Any help would be appreciated.
... View more