Monitoring Splunk

Delete busy jobs in the cli

chris
Motivator

Hi

We recently had the situation, where the web interface was responding slowly and we had a high load on the system caused by splunk processes.

The following message was in splunkd.log:

WARN  DispatchCommand - The system is approaching the maximum number of historical searches that can be run concurrently. current=55 maximum=68  

We think this could be because the searches running concurrently cause the high load.

Is there a way to use the CLI to display how resource intensive jobs/searches are that are currently running and is there a way to kill/cancel/delete resource intensive jobs?

Thanks

Chris

0 Karma
1 Solution

Dark_Ichigo
Builder

To search for the Current jobs:

ps -ef | grep "splunkd search"

http://www.splunk.com/base/Documentation/latest/Admin/ManagejobsintheOS

To terminate, just kill the proccess ID

View solution in original post

Dark_Ichigo
Builder

To search for the Current jobs:

ps -ef | grep "splunkd search"

http://www.splunk.com/base/Documentation/latest/Admin/ManagejobsintheOS

To terminate, just kill the proccess ID

Dark_Ichigo
Builder

No worries 🙂

0 Karma

chris
Motivator

Thanks, it's: ps -ef | grep "splunkd search" can you change that in the response and I'll accept it.

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...