Splunk Search

Search to Identify when a host stops sending logs to Splunk

Makinde
New Member

Hello,

I have this search string to identify hosts that have stopped sending logs to Splunk, however the search string below identifies every hosts that has ever stopped sending logs, however I want only hosts that have not sent any logs in the past 3 days. What do I need to change in this search string to get that number?

|metadata type=hosts | eval age = now() - lastTime | search age > 86400 | sort age d | convert ctime(lastTime) | fields age,host,lastTime

Tags (1)
0 Karma
1 Solution

twinspop
Influencer

I would probably use tstats instead of metadata, and just set the time selector to last 3 days:

| tstats latest(_time) as lastTime by host | 
eval age=now()-lastTime | 
search age > 86400 | 
sort age d | 
convert ctime(lastTime) | 
fields age,host,lastTime

Metadata can yield unexpected results when you set a timeframe.

EDIT: changed to days from hours. Doh!

View solution in original post

twinspop
Influencer

I would probably use tstats instead of metadata, and just set the time selector to last 3 days:

| tstats latest(_time) as lastTime by host | 
eval age=now()-lastTime | 
search age > 86400 | 
sort age d | 
convert ctime(lastTime) | 
fields age,host,lastTime

Metadata can yield unexpected results when you set a timeframe.

EDIT: changed to days from hours. Doh!

Makinde
New Member

Hi Twinspop,

Thanks for this new search, it appears to work better than the Metadata. Just curious, If I understand properly, this search looks at logs as far back as I specify in my time selector and identifies hosts that haven't reported in the time specified in the search age criteria (search age > 86400) in this case more than a day?

Is that what this search does?

0 Karma

twinspop
Influencer

Correct. The tstats command will follow your time restraint. This command will initially find all hosts that have logged any data in the last 3 days in any index. The filtering will then only show those that stopped more 86400 seconds ago.

0 Karma

muebel
SplunkTrust
SplunkTrust

Hi Makinde, something like this should work (could probably drop the 86400 down closer to now to be more inclusive)

|metadata type=hosts | eval age = now() - lastTime | search age > 86400 AND age < 259200 | sort age d | convert ctime(lastTime) | fields age,host,lastTime

Please let me know if this answers your question!

Makinde
New Member

Hi Muebel,

The new search definitely makes a change in my results however I noticed it doesn't identify hosts that stopped sending logs older than 3 days ago. So say a host stopped sending logs last month and it hasn't sent any logs up until now that won't show up in this search result.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...