All Apps and Add-ons

check process is still running on each server

grahamrb
Explorer

I'm trying to figure out a way to verify that a process is running on each server. We have a clustered environment with 4 servers running WebSphere application server and as such I want to confirm that the cluster is healthy at the process level.

This is the closest I've gotten:

 sourcetype="ps" "SOME_UNIQUE_PROCESS_NAME" | timechart count by host

Pushing this into an area graph gets close except during some intervals the ps source isn't sampled and therefore returns 0 results instead of 1. It also isn't very robust as as it relies on the span of each sample of the timechart.

I'm using the Splunk for Unix and Linux app which is where the sourcetype of 'ps' comes from.
http://apps.splunk.com/app/273

grahamrb
Explorer

I think I figured it out, could someone validate if this is the most efficient way to do this:

 sourcetype="ps" | eval processexists=if(match(_raw, "SOME_UNIQUE_PROCESS_NAME"), 1, 0) | timechart span=1m avg(processexists) by host

As long as you know you're going to get more than one event from ps within the span of 1 minute it should work. The only thing that could get dodgy is it'll drop down to 0.5 between going from 1 and 0 if you are sampling every 30 seconds.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...