Splunk Search

Writing a search that will catch sourcetypes that have logged in x amount of time

brent_weaver
Builder

We are wokring on coming up with a methd to detect data that stops coming in based on sourcetype. I believe I will want to look at the metadata like:

|metadata index=index_name type=sourcetypes 

This yields a nice list (based on index of course). How do I structire this metadata query to pick out sourcetypes that have not reported in in say 1 day? I would assume I woudl leverage the epoch timestamp and do some simple math?

Any help is MUCH apprecaited!

Tags (1)
0 Karma
1 Solution

dineshraj9
Builder

You can take a difference of current time and lastTime field and check if its greater than a day -

 | metadata index=index_name type=sourcetypes | eval diff=now() - lastTime | where diff > 86400 | convert ctime(lastTime)

Similar query be created to identify hosts which have stopped reporting (even if no non-internal logs are generated internal logs are continuously read from hosts) -

 | metadata type=hosts | eval diff=now() - lastTime | where diff > 86400 | convert ctime(lastTime)

View solution in original post

0 Karma

dineshraj9
Builder

You can take a difference of current time and lastTime field and check if its greater than a day -

 | metadata index=index_name type=sourcetypes | eval diff=now() - lastTime | where diff > 86400 | convert ctime(lastTime)

Similar query be created to identify hosts which have stopped reporting (even if no non-internal logs are generated internal logs are continuously read from hosts) -

 | metadata type=hosts | eval diff=now() - lastTime | where diff > 86400 | convert ctime(lastTime)
0 Karma

brent_weaver
Builder

Thank you for the great info, and they both worked perfectly. One last question, is tstats any more efficient? We have also been playing with:

| tstats latest(_time) as latest where index=index* by sourcetype | where latest < relative_time(now(), "-60m") | eval "last seen"=strftime(latest,"%m/%d/%y %H:%M:%S") | fields – latest

Which is faster? I would suspect meta, but if I knew I woudl not bother you with this question! 🙂

Thanks!

0 Karma

dineshraj9
Builder

Metadata query are not time dependent, but tstats queries are. Also if you want to find a sourcetype that had stopped reporting last week, then the time range for tstats command has to be greater than 7days.
Metadata queries are faster in this respect, but they can also be slow if you have huge number of results(sourcetypes or hosts).

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...