Monitoring Splunk

Configure splunkweb to stop when splunkd crashes

aengelstad
Engager

We've unfortunately had a few instances now where for various reasons the splunkd processes crashes or otherwise stops. Unfortunately the splunkweb process keeps running and presenting a login page HTTP/200 so our load balancer keeps it in the pool.

Ideally there would be a config to have splunkweb shutdown then the splunkd process returns HTTP4XX/5XX or timeouts however I can't see that anywhere.

Worst case I could write a shell script and cron it, but I'm trying to avoid that if I can.

Obviously the easy answer would be to stop splunkd from crashing, but I figure it's better to plan for failure.

Tags (3)
0 Karma

MuS
Legend

Hi aengelstad,

try to use MartinMcNutt's suggestion and have the load balancer also test on port 8089 and if this fails remove port 8000 from the load balancer pool. Other solution would be the script cron job that you suggested.

cheers, MuS

0 Karma

MartinMcNutt
Communicator

What about doing the health check against port 8089 too? Not sure what load balancer you have but I would give high priority to the check on 8089 then 8000.

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...