Alerting

Dynamically updated alert search

hartfoml
Motivator

I am running a port monitoring app using NMAP that record system availability

If this stops it makes my system availability stats wrong

I would like to create and alert that will tell me if this is not working.

Not too hard but I would like the alert to be dynamic and adjust for the number of systems I am monitoring.

Right now I have 60 systems monitored every minute so ( 60*60 ) or 3600 events per hour

So I can set an alert for less than 3600 event but if I add a system, the alert should go up or ( 61*60 ) trigger = 3660 per hour

How do I write the alert to have dynamic triggers?

Tags (2)
0 Karma
1 Solution

kristian_kolb
Ultra Champion

If your nmap application writes one event per monitored port per minute, and you'd like to find out if the port monitoring app itself has crashed, you could simply have a scheduled search that will detect if the logs stop coming in completely.

<your source/sourcetype> earliest=-1m 

with the alert triggering set on the condition "number of events = 0"

Have I understood your problem correctly? Is this a viable solution?

/kristian

View solution in original post

kristian_kolb
Ultra Champion

If your nmap application writes one event per monitored port per minute, and you'd like to find out if the port monitoring app itself has crashed, you could simply have a scheduled search that will detect if the logs stop coming in completely.

<your source/sourcetype> earliest=-1m 

with the alert triggering set on the condition "number of events = 0"

Have I understood your problem correctly? Is this a viable solution?

/kristian

kristian_kolb
Ultra Champion

You are most welcome

/k

0 Karma

hartfoml
Motivator

I'm sorry I was looking for one kind of answer and you gave me the right one. It was not what I expected so I thought it must be wrong. I was looking for a way to alert on the input not the monitoring. I have setup an alert like you have suggested above and this will do what I wanted. Thanks for the help I really appreciate it.

0 Karma

kristian_kolb
Ultra Champion

Sorry, I thought you wanted to monitor availability in splunk, and also monitor the status of the monitoring application itself - and that it was the latter problem you wanted help with. I still don't know really what you want to perform.

/k

0 Karma

hartfoml
Motivator

Thanks for answering I really appreciate your participation and trying to help. I can't monitor the log inputs because some of the clients are quit for longer than a minute and I wanted to get a one minute interval to monitor availability.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...