What settings do I need to change to regain permissions to read metrics.log?

[seema2502]

Hi Splunkers,

I have lost permissions to read the metrics logs.
I have asked the platform team and they have updated me with the message:
"We changed permissions manually on affected file to splunk:splunk metrics.log also on metrics.log.1 and metrics.log.2. We cannot confirm, after next rotate correct permission, because rotation mentioned "metrics" fields is not under OS management but under APP. Please check settings of rotation from your side."

Please advise where I need to change the settings to get the permission permanently.

Thanks,
Seema

[dbanerjee17]

I am not sure whether your issue had got resolved. I have got the same issue now. Three logs have stopped logging since April 17, 2018. They are splunkd.log, metrics.log and splunk-access.log. Also, now I am getting access denied error while restarting splunk and also KV store failed due to access denied error for /kvstore/mongo/_tmp directory. Our splunk instance runs with splunk user. I by mistake started with root user. After realizing it, I stopped and restarted splunk with splunk user. Just after that, the problem started. All the three logs files and_tmp directory mentioned above are currently running with root. All others are running with splunk user. I am not sure whether the above mentioned log files and _tmp direct should be with splunk ownership. I have struggling with issue for the past few days. Any help on this will be highly appreciated.

[dbanerjee17]

Thanks for your response; I was hesitant to try chown as I was not sure whether the mentioned files also should have splunk ownership. When I ran the top command, I found that all the splunk related processes were running with splunk user; also the splunk directory was with splunk:splunk ownership. I will stop and will give a try.

[richgalloway]

Everything under $SPLUNK_HOME (and $SPLUNK_DB, if it's not under $SPLUNK_HOME) should be owned by the user running Splunk.

[richgalloway]

@ dbanerjee17 You're adding on to a three-year-old question. You'll have better chances at getting a helpful response if you post a new question.
That said, when you mistakenly run Spunk as root it's critical to run chown -R splunk:splunk /opt/splunk before restarting Splunk as user splunk.

[lguinn2]

The logs are rotated by Splunk. What user is Splunk running as? The best practice is for Splunk to be running as a privileged user rather than root or administrator. However, on Linux, if you have restarted Splunk from the root account, future log files may be owned by root.

So I would check to see what user owns the log files and make appropriate adjustments...

[seema2502]

Hi Lguinn,

Thanks for the update, Splunk is running as a privileged user but due to some change going on this server i lost the access on the log files and root owned the access.
now what adjustment i can do from splunk end to get the access back for privileged user.

Thanks,
Seema

[lguinn2]

If you are running splunk on linux, the following commands will work. This solution assumes that Splunk is installed at /opt/splunk and that the privileged user is named puser:

cd /opt
chown -R puser splunk/

The above commands will need to be run by root.

If you are running on Windows, I don't know how to do this easily. You just need to change the ownership of the entire directory tree where Splunk is installed, so it is all owned by the Splunk user.