Splunk Cloud Platform

Splunk stop with SplunkLightForwarder turned on takes.....forever! Why?

MasterOogway
Communicator

I have SplunkLightForwarder turned on for AIX and Linux (Suse) and when initiating a Splunk stop it takes 5-10 minutes. The only reason I need to stop/start is because of the (3) splunkd daemons running (one too many) which in turn doesn't allow a newly installed Deployclient to query/copy/install the new .conf files pulled from the Deployment server. (is this a bug?)

So....Big picture....why does "Splunk stop" take so long?

pstein

Tags (1)
0 Karma

bbingham
Builder

I just experienced this and recently sorted out my issue with support.

Our issue stemmed from what license file was installed on the forwarder. As you stated above, when upgrading an old forwarder, shutdown times would take forever and there were 3 daemons started. After some exploration, we found the enterprise trial license was set as the current license on each machine. Each splunk license is also shipped with a forwarding license, simply replacing the splunk.license file under $SPLUNK_HOME/etc/ with the splunk-forwarder.license resolved the shutdown issue and the extra daemon.

To give it a shot, simply log on to the forwarder, cd $SPLUNK_HOME/etc cp splunk-forwarder.license splunk.license

restart.

Let me know if this worked for you as well!

bbingham
Builder

Sure, SPL-31257

0 Karma

MasterOogway
Communicator

lephino....do you have a SPL number I can search against to see if this was fixed in 4.1.5?

0 Karma

bbingham
Builder

After trouble shooting with support, we did come across a known bug in 4.1.4 with the restart hanging up the forwarders. It's slated to be fixed in 4.1.5

0 Karma

bbingham
Builder

After further investigation, I can still recreate the 3 daemon pids by pushing the SplunkLightForwarder app to a remote machine and have the forwarder set to restart. Will update this thread if we find a solution.

0 Karma

Branden
Builder

I sometimes have the same problem as you, and we're also running AIX. I'm curious to see what responses you get to this post.

I don't know if this is related or not (probably not), but I noticed the 5-10 minute stop times on Splunk didn't happen as often after we moved to AIX 6.1. That's probably a coincidence, but I thought I'd point it out.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...