What are my options to specify the user to start splunk service as on linux

[yannK]

I want to run splunk on linux on a cluster as non root user, I found several ways to change the user.
( boot-start, the init.d/splunk service, the splunk-launch.conf )

What are the advantages of each method, and the behavior with restarts, service restart and rolling restarts ?

[yannK · Accepted solution]

There are several ways on linux to specify the user to start the splunk process.

By example to start splunk as the dedicated user "splunk"
( We assume that the user does exists in your system. )

• A - the user can defined in the splunk launcher options
  in /opt/splunk/etc/splunk-launch.conf under SPLUNK_OS_USER
  example in /opt/splunk/etc/splunk-launch.conf SPLUNK_OS_USER=splunk

  • - What if the user specified does not exists in the system, then splunk will fail to start.

• B - the user can defined as a system service (/etc/init.d/splunk under SPLUNK_USER )
  setup by using the splunk enable boot-start command http://docs.splunk.com/Documentation/Splunk/latest/Admin/ConfigureSplunktostartatboottime
  example : in /etc/init.d/splunk SPLUNK_USER=splunk
  Then start splunk as a service, or at boot time : service splunk start

  • - If A and B are used, A will have the last word.
  • - What if the user specified does not exists in the system, then splunk will fail to start.
  • C - otherwise, the user is the identity of the user doing the CLI manual start/restart sudo su splunk - /opt/splunk/bin/splunk start
  • - If A and C are used, A will have the last word.
  • - If B and C are used, C will have the last word.

---

Behavior in case of a restart or rolling restart :

By example if you are remotely triggered rolling restart (in the case of an indexers in a cluster, or a deployment client)

• A - with the splunk-launch.conf , then splunk will restart as this user defined under SPLUNK_OS_USER
• B - with the /etc/init.d/splunk, it will have no effect in case of restart, as splunk will restart as the user it was already running as
• C - by default splunk will restart as the user it was already running as

So If you are using A, it will prevail, but B and C will not have an effect on a splunk restart.

---

What are the possible side consequences of switching user ?

If you were running splunk as one user, then restarted splunk as another user, you may encounter some problems.

• If you were running splunk as root, then some internal splunk file are owned by root. As a consequence if you restart as the user "splunk", it may not be able to read the files, and will fail to start or throw errors.
• If you ran a splunk btool command as root, while splunk was running as another user, the command may create a log file owned by root ($SPLUNK_HOME/var/log/splunk/btool.log), and it will throw errors next time you run a btool as the user "splunk", or when splunk will try to index the internal log file.
  • If you were running splunk process as "splunk" and switched to "root", it should work. But be careful if you switch back.

The solutions are to change the ownership of the files.
Example on linux

/opt/splunk/bin/splunk stop
sudo chown -R splunk: /opt/splunk/
/opt/splunk/bin/splunk start   

[goodsellt]

Would you be able to add some comments regarding behavior when performing an OS/System level reboot?

[yannK]

If you reboot your server, and have setup a boot-start command (option B) for splunk , then splunk will automatically start as a service.
It will try to start using the user defined in /etc/init.d/splunk under SPLUNK_USER
(if the SPLUNK_USER was not defined, it will try root)

However, if you also had used the option A, so enforce a different user in /opt/splunk/etc/splunk-launch.conf under SPLUNK_OS_USER, then it will actually switch to this user.

So if SPLUNK_USER and SPLUNK_OS_USER are different, SPLUNK_OS_USER wins
while if no SPLUNK_OS_USER is defined, it will use the user from the service (SPLUNK_USER)

[goodsellt]

Thanks! This is all fantastic information!