Alerting

Alert when service is down on multiple hosts

fsrodriguez
New Member

I have the Splunk Add-On for Linux and Unix installed which enables the "PS" command. How can I monitor the tomcat service or some other service on multiple hosts for 5 minutes?

This is what I have so far:

host="server-*" source="ps" tomcat

I would like to trigger an alarm whenever the tomcat service has been down for more than 5 minutes on any of the hosts that the query finds.

0 Karma

Yunagi
Communicator

Try the following search:

host="server-*" source="ps" process_name="tomcat" | dedup host | eval lastseen=now()-_time

You might need to change process_name="tomcat" to suit your needs. You also might want to add "index=..." This will make your searches faster.

Save this search as an alert with the custom trigger condition: lastseen>300. The time range should be several hours, e.g. last 24 hours.

0 Karma

fsrodriguez
New Member

when you say "You also might want to add "index=...""....

This means setting up a monitor this way:

splunk add monitor /opt/tomcat/logs/catalina.out  -index tomcat

Right?

and then do:

 host="server-*" source="tomcat" process_name="tomcat" | dedup host | eval lastseen=now()-_time
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...