Getting Data In

Should I delete "/etc/init.d/splunk" when I update to 7.2.3 from before 7.2.2?

yutaka1005
Builder

When I update Splunk to ver 7.2.3 from before ver 7.2.2(like 6.6.x) and enable boot-start, unit file is created under "/etc/systemd/system/", but "/etc/init.d/splunk" is still remaining.

In such a case, what is the recommended method?
Just by deleting "/etc/init.d/splunk" does not matter?

I can't find description about case like this in install manual, so if someone knows about it, please tell me.

1 Solution

dd_msearles
Path Finder

The fact Splunk snuck this change into a maintenance release is worth noting, I feel like they are using those of use you use the latest and greatest as guinea pigs.

In addition to removing /etc/init.d/splunk I would remove:

user@host:~# ls -l /etc/rc.d/rc?.d/[KS]90splunk
lrwxrwxrwx. 1 root root 16 Oct 23  2017 /etc/rc.d/rc2.d/S90splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root 16 Oct 23  2017 /etc/rc.d/rc3.d/S90splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root 16 Oct 23  2017 /etc/rc.d/rc4.d/S90splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root 16 Oct 23  2017 /etc/rc.d/rc5.d/S90splunk -> ../init.d/splunk

It would be nice if the enable boot command cleaned this up, however I can understand for backwards compatibility with scripting and automation why this is unlikely to happen. I think they will just draw a line in the sand for 7.2.2 and up.

View solution in original post

0 Karma

bandit
Motivator

Summary of the issue:
Splunk 6.0.0 - Splunk 7.2.1 defaults to using init.d when enabling boot start
Splunk 7.2.2 - Splunk 7.2.9 defaults to using systemd when enabling boot start
Splunk 7.3.0 - Splunk 8.x defaults to using init.d when enabling boot start

systemd defaults to prompting for root credentials upon stop/start/restart of Splunk

Here is a simple fix if you have encountered this issue and prefer to use the traditional init.d scripts vs systemd.

Splunk Enterprise/Heavy Forwarder example (note: replace the splunk user below with the account you run splunk as):

sudo /opt/splunk/bin/splunk disable boot-start
sudo /opt/splunk/bin/splunk enable boot-start -user splunk -systemd-managed 0

Splunk Universal Forwarder example (note: replace the splunk user below with the account you run splunk as):

sudo /opt/splunkforwarder/bin/splunk disable boot-start
sudo /opt/splunkforwarder/bin/splunk enable boot-start -user splunk -systemd-managed 0
0 Karma

dimrirahul
Explorer

How systemd tries to look for a configuration to start a service is as follows, first it tries to look for unit file in /etc/systemd/system or other well known paths to it, if its unable to find such a file then it tries to look for traditional init.d/ styled files and tries to do some mapping to run it correctly.
So if you were to just upgrade splunk and not run enable boot-start you would probably be fine.
If you wanted to run boot-start but still wanted to preserve init.d behavior add '-systemd-managed 0'
If you wanted to run boot-start and enforce a unit file to be written to /etc/systemd/system irrespective of if your system is actually running systemd use '-systemd-managed 1'.
If you wanted to splunk make an informed guess (which is mostly correct) just dont specify this option. Splunk will look at system and then try to do the right thing that is on systemd systems install a unit file and on init.d type systems put a corresponding startup script in /etc/init.d/ directory.

0 Karma

dd_msearles
Path Finder

The fact Splunk snuck this change into a maintenance release is worth noting, I feel like they are using those of use you use the latest and greatest as guinea pigs.

In addition to removing /etc/init.d/splunk I would remove:

user@host:~# ls -l /etc/rc.d/rc?.d/[KS]90splunk
lrwxrwxrwx. 1 root root 16 Oct 23  2017 /etc/rc.d/rc2.d/S90splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root 16 Oct 23  2017 /etc/rc.d/rc3.d/S90splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root 16 Oct 23  2017 /etc/rc.d/rc4.d/S90splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root 16 Oct 23  2017 /etc/rc.d/rc5.d/S90splunk -> ../init.d/splunk

It would be nice if the enable boot command cleaned this up, however I can understand for backwards compatibility with scripting and automation why this is unlikely to happen. I think they will just draw a line in the sand for 7.2.2 and up.

0 Karma

dd_msearles
Path Finder

My search above is not complete, there's files prefixed with 90 and 60. A bit more generic now:

user@host:~# ls -l /etc/rc.d/rc?.d/[KS][0-9][0-9]splunk
lrwxrwxrwx. 1 root root  16 Oct  5 17:39 /etc/rc.d/rc0.d/K60splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root  16 Oct  5 17:39 /etc/rc.d/rc1.d/K60splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root  16 Oct  5 17:39 /etc/rc.d/rc2.d/S90splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root  16 Oct  5 17:39 /etc/rc.d/rc3.d/S90splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root  16 Oct  5 17:39 /etc/rc.d/rc4.d/S90splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root  16 Oct  5 17:39 /etc/rc.d/rc5.d/S90splunk -> ../init.d/splunk
lrwxrwxrwx. 1 root root  16 Oct  5 17:39 /etc/rc.d/rc6.d/K60splunk -> ../init.d/splunk

Has anyone tried using chkconfig to see if that can be used to tidy up? (chkconfig --list splunk, chkconfig --del splunk)?

0 Karma

yutaka1005
Builder

Thank you for answer!

I found below line in manual.

If you have previously run enable boot-start in a version of Splunk Enterprise prior to 7.2.2, remove the splunk start script located by default in /etc/init.d.

https://docs.splunk.com/Documentation/Splunk/7.2.2/Admin/RunSplunkassystemdservice

0 Karma

gjanders
SplunkTrust
SplunkTrust

I've added feedback to the documentation page to request they mention the removal of the rc.d links as well, otherwise the init.d will attempt to find the splunkd script when changing runlevels...

0 Karma

dkeck
Influencer

Hi,

just saw this, its not talking about deleting etc/init.d but maybe you can take some useful information out of it. https://www.duanewaddle.com/splunk-7-2-2-and-systemd/

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...