Splunk Search

How to create a Splunk alert to trigger when Tomcat is down?

splunker9999
Path Finder

Hi,

We have scenario to create an alert for tomcat to trigger an alert when tomcat is down.

Based on our tomcat logs, it gives PID for every 30secs when ever it is up.
If it is down it wont trigger any event for that 30 sec interval.

We need to set up an alert to trigger by host ,if any of the host has not have an entry for 30secs period.

Below is basic search for it:

 index=index1  source=ps host=host1* OR host=host2* apache-tomcat|table host pid _time
0 Karma

twinspop
Influencer

Find hosts down for 5 minutes:

index=index1  source=ps host=host1* OR host=host2* apache-tomcat | 
stats latest(_time) as latest by host | 
eval age=now()-latest | where age>300

splunker9999
Path Finder

Hi, This would works when host is down.

Here we are checking for pid in the events: We will get events for every 30secs when ever Tomcat is up and running.

When ever server is down, it wont trigger any events to splunk
Ex:
If Tomcat is stopped around 3.47.30.000, we wont get any events from 3.47.30.000 to until it is up.

So we need to customise our search , in such away that if search find no events from any particular host ..we need to give status as down. By default when ever search returns events which have pid, then status should be up.

If we have search query for this , we can give condition like|where status=down

Thanks

0 Karma

twinspop
Influencer

Change the stats command to include pid: ... as latest by host pid ?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...