Getting Data In

How to monitor Server hung state on both Windows and Linux using Splunk?

ansif
Motivator

Can anyone help me to know the possibility of monitoring server hung state using Splunk?

0 Karma
1 Solution

woodcock
Esteemed Legend

This app was just posted yesterday and probably does everything you need (and if not the author will surely be very responsive):
Unified Forwarder Monitoring App for Splunk: https://splunkbase.splunk.com/app/3805/

View solution in original post

woodcock
Esteemed Legend

This app was just posted yesterday and probably does everything you need (and if not the author will surely be very responsive):
Unified Forwarder Monitoring App for Splunk: https://splunkbase.splunk.com/app/3805/

ansif
Motivator

This is working and it has the module what @niketnilay suggetsed.

Thanks @woodcock and @niketnilay

0 Karma

niketn
Legend

@ansif, we had an issue on our Windows servers which used to hang during scheduled reboot and would not shut down. While servers used to respond to ping request, almost all services including the Remote Login used to stop functioning. Obviously Splunk's splunkd service also used to stop. Which implied that Splunk Universal Forwarder on such Windows machines (Splunk Deployment Client) would stop pinging Deployment Server. We could do either one of the following to check for such events:

1) Check the Deployment Console in DMC to identify the deployment clients which have not pinged recently.

2) Use REST API to get the deployment clients which have not pinged recently (following example if for last 5 min):

| rest /services/deployment/server/clients
| table name lastPhoneHomeTime
| where lastPhoneHomeTime>300

PS: DMC also uses the same REST API: https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTdeploy#deployment.2Fserver.2Fclients

If you are already indexing some stats from these Windows and/or Linux machines(even if Splunk's internal logs from these hosts) there may be several other options as well. Refer to the following Answer thread: https://answers.splunk.com/answers/592278/query-for-splunkd-status.html#answer-593319

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

ansif
Motivator

@niketnilay : Could you please help me with a solution on below:

Our deployment server is at customer site,is there any option to create an inputs.conf definition?

0 Karma

woodcock
Esteemed Legend

Why do you post so many answers as comments, @niketnilay?

0 Karma

niketn
Legend

@woodcock, 😄 for several reasons but I mostly try to keep questions as unanswered so that others can assist with their inputs as well. Other cases when I require further clarification or am not sure of the solution I provide then also I post as a comment to hear back. Keeping as unanswered leaves the question open for others in case the poster responds back and I miss.

This is complete different case though 🙂 Ansif and I work at same company, so I can follow up with him at work 🙂

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...