All Apps and Add-ons

Does changing Splunk from Local user account to Domain account will cause any errors ?

damode
Motivator

I had installed Splunk as Local account. However, for installing DB Connect, the documentation states that the Splunk services needs to run as domain user. http://docs.splunk.com/Documentation/DBX/3.1.1/DeployDBX/Installdatabasedrivers#Install_the_SQL_Serv...

After changing the "Log on" as setting from the Local System account to that of the logged on domain user. I got the below messages on Search Head after restart.

Failed to start KV Store process. See mongod.log and splunkd.log for details.
31/10/2017, 10:15:10
KV Store changed status to failed. KVStore process terminated.
31/10/2017, 10:15:01
KV Store process terminated abnormally (exit code 100, status exited with code 100). See mongod.log and splunkd.log for details.

Please help

0 Karma
1 Solution

MuS
SplunkTrust
SplunkTrust

Did you check mongod.log and/or splunkd.log to what the actual error is?

View solution in original post

0 Karma

jkat54
SplunkTrust
SplunkTrust

Is reinstalling splunk an option? If so, uninstall,
Reinstall, select the advanced options during install and define the account splunk will run as during the install. It’s either that or recursively applying ownership (perhaps wrongly) to the splunk folder and all its sub folders.

The install method is probably the easiest.

Any reason why you have to run as a domain account? It’s best practice to use a MSA but not a requirement of dbconnect. You can just as easily use sql authentication and a local sql user.

0 Karma

MuS
SplunkTrust
SplunkTrust

Did you check mongod.log and/or splunkd.log to what the actual error is?

0 Karma

damode
Motivator

Splunkd.log has the same above mentioned errors. Plus some more like,
KVStoreConfigurationProvider - Could not get ping from mongod.
10-31-2017 10:15:10.505 +1100 ERROR KVStoreConfigurationProvider - Could not start mongo instance. Initialization failed.
10-31-2017 10:15:10.505 +1100 ERROR KVStoreBulletinBoardManager - Failed to start KV Store process. See mongod.log and splunkd.log for details.
ERROR MongodRunner - mongod exited abnormally (exit code 100, status: exited with code 100) - look at mongod.log to investigate.

Below is from mongod.log,
2017-10-30T23:15:00.990Z I STORAGE [initandlisten] exception in initAndListen: 98 Unable to create/open lock file: C:\Program Files\Splunk\var\lib\splunk\kvstore\mongo\mongod.lock errno:5 Access is denied.. Is a mongod instance already running?, terminating

0 Karma

MuS
SplunkTrust
SplunkTrust

This is your problem Unable to create/open lock file: C:\Program Files\Splunk\var\lib\splunk\kvstore\mongo\mongod.lock errno:5 Access is denied

Check the file permission or delete the file and let Splunk create it with a restart.

0 Karma

damode
Motivator

I changed owner of both files to domain user. Not getting that error any more.
Thanks for your help!

0 Karma

MuS
SplunkTrust
SplunkTrust

Converted to an answer, feel free to accept it 😉

0 Karma

MuS
SplunkTrust
SplunkTrust

Also check this post https://answers.splunk.com/answers/490134/how-to-resolve-issues-with-mongod-startup-such-as.html even this was a linux instance, but it hints to permission problem. Or this one https://answers.splunk.com/answers/490134/how-to-resolve-issues-with-mongod-startup-such-as.html mentions a manual mongoDB restart which removed a stale lock file.

0 Karma

damode
Motivator

Thanks MuS.

I had actually already checked that post and confirmed that my domain account had had full permission on splunk.key. I am not sure if its the equivalent of setting chmod 600 /opt/splunk/var/lib/splunk/kvstore/mongo/splunk.key in Linux.

I also noticed the owner of this was "System". I have just changed it to domain user. Not sure if it will work.

Your second link is same as first one. Can you please post the second link ? 🙂

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...