Getting Data In

monitor that proper amount of events are coming in per host

jfraiberg
Communicator

I am trying to build an app that will set a baseline per host of event count that will alert me when a hosts event count has dropped by a certain % per hour.

Has anyone tried something like this? I find that monitoring the agent alone is not enough and would like to know if a host is sending less events than normal.

Any ideas or advice would be welcome 🙂

Tags (1)
0 Karma
1 Solution

ziegfried
Influencer

This search would compare the count by host of the previous hour with the current hour and filter those where the count dropped by more than 10%:

earliest=-2h@h latest=@h | stats count by date_hour,host | stats first(count) as prev last(count) as cur by host | eval cur_pct=(cur/prev)*100 | where cur_pct<90

View solution in original post

ziegfried
Influencer

This search would compare the count by host of the previous hour with the current hour and filter those where the count dropped by more than 10%:

earliest=-2h@h latest=@h | stats count by date_hour,host | stats first(count) as prev last(count) as cur by host | eval cur_pct=(cur/prev)*100 | where cur_pct<90

jfraiberg
Communicator

This totally worked, thanks! The search itself took forever though and took alot of resources to run. I will play with it some more. thanks!!!

0 Karma

jameswatts
Explorer

You can speed the search up significantly with tstats:
|tstats count by host, _time span=1h| stats first(count) as prev last(count) as cur by host | eval cur_pct=(cur/prev)*100 | where cur_pct<90

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...