Getting Data In

How to monitor a windows service, send an alert and restart the service?

mtoddsmith
Engager

How can we to monitor various windows services and send alerts when they are down and optionally attempt to restart the service via splunk.

Tags (3)
1 Solution

Chubbybunny
Splunk Employee
Splunk Employee

use the sc. exe command to check on the status of each service.

DESCRIPTION:
SC is a command line program used for communicating with the
NT Service Controller and services.

C:\Documents and Settings\chubbybunny.hare>sc.exe query SplunkForwarder

SERVICE_NAME: SplunkForwarder
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE,NOT_PAUSABLE,ACCEPTS_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

craft a splunk search for 'STATE STOPPED'

C:\Documents and Settings\chubbybunny.hare>sc.exe query SplunkForwarder

SERVICE_NAME: SplunkForwarder
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 1  STOPPED
                                (NOT_STOPPABLE,NOT_PAUSABLE,IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

add an Alert action to 'Run a script' to start the service

C:\Documents and Settings\chubbybunny.hare>sc.exe start SplunkForwarder

SERVICE_NAME: SplunkForwarder
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 2  START_PENDING
                                (NOT_STOPPABLE,NOT_PAUSABLE,IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x7d0
        PID                : 4000
        FLAGS              :

upVote the Chubbybunny if it helps!

(\__/)
(='.'=)
(")_(")

View solution in original post

dwaddle
SplunkTrust
SplunkTrust

We accomplish half of this via WMI. Add this to wmi.conf and push it out via deployment server (or however you push out configs)

[WMI:Services]
interval = 60
disabled = 0
index = default
wql = select Name, ProcessId, Caption, DisplayName, State, Status, StartName, SystemName from Win32_Service

Once you're collecting data, then searching/alerting on services that are not in the right status is pretty simple.

noy72
New Member

I have added the additional displaynames to my WMI.conf file on the SCCM forwarder, I have veified that the WMI namespace on SCCM has theapropriate permissions and I have verified that Splunk is receiving WMI data. Can someone help me out with a query to veify the status of SCCM specific services?
Thank you
Ron Jones

0 Karma

noy72
New Member

After verifing the above, al searches started working. I apreciate the assistance.
Ron Jones

0 Karma

Chubbybunny
Splunk Employee
Splunk Employee

use the sc. exe command to check on the status of each service.

DESCRIPTION:
SC is a command line program used for communicating with the
NT Service Controller and services.

C:\Documents and Settings\chubbybunny.hare>sc.exe query SplunkForwarder

SERVICE_NAME: SplunkForwarder
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE,NOT_PAUSABLE,ACCEPTS_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

craft a splunk search for 'STATE STOPPED'

C:\Documents and Settings\chubbybunny.hare>sc.exe query SplunkForwarder

SERVICE_NAME: SplunkForwarder
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 1  STOPPED
                                (NOT_STOPPABLE,NOT_PAUSABLE,IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

add an Alert action to 'Run a script' to start the service

C:\Documents and Settings\chubbybunny.hare>sc.exe start SplunkForwarder

SERVICE_NAME: SplunkForwarder
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 2  START_PENDING
                                (NOT_STOPPABLE,NOT_PAUSABLE,IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x7d0
        PID                : 4000
        FLAGS              :

upVote the Chubbybunny if it helps!

(\__/)
(='.'=)
(")_(")
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...