Splunk Dev

How do I prevent Introspection Generator to read information about non splunk process when hidepid activated on /proc ?

maraman_splunk
Splunk Employee
Splunk Employee

Hi,

my splunk is running as splunk user on a linux system where the admin has secured the OS by using hidepid=1 on /proc (see https://ubuntuforums.org/showthread.php?t=2173093 and https://www.kernel.org/doc/Documentation/filesystems/proc.txt)

As a consequence, splunkd.log is filled with these error messages :
ERROR IntrospectionGenerator:resource_usage - RU - Fail to readlink(2) /proc/nnnn/exe: Operation not permitted where nnnn is a pid from a process not run by splunk
This is repeated for each pid so generate a lot of noise.

I would like to tell Introspection to only look at it's own pid in that case or not produce error message for this.

Any idea how to do this ?

0 Karma

ipfyx
Engager

Hi,

you can also add the splunk group gid to the fstab ($ id splunk_user) :
proc /proc proc rw,nosuid,nodev,noexec,relatime,gid=<splunk_gid>,hidepid=1 0 0

According to man proc :

       gid=gid (since Linux 3.3)
              Specifies the ID of a group whose members are authorized
              to learn process information otherwise prohibited by
              hidepid (i.e., users in this group behave as though /proc
              was mounted with hidepid=0).  This group should be used
              instead of approaches such as putting nonroot users into
              the sudoers(5) file.

 

maraman_splunk
Splunk Employee
Splunk Employee

As a workaround, I completely disabled the generator for resource usage
in server.conf

[introspection:generator:resource_usage]
disabled=true

this stop the error message flood but that will also disable all related stats in the monitoring console....

Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...