All Apps and Add-ons

Why is the S.o.S - Splunk on Splunk auditd.service giving errors when running rlog.sh in the Splunk Add-on for Unix and Linux?

BlueSocket
Communicator

I am getting errors in SOS as below. I think that the Splunk_TA_nix app is causing them when running rlog.sh:

Redirecting to /bin/systemctl status  auditd.service
type=USER_ACCT msg=audit(10/19/2015 15:50:01.877:15323) : pid=11620 uid=root auid=unset ses=unset subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting grantors=pam_access,pam_unix,pam_localuser acct=root exe=/usr/sbin/crond hostname=? addr=? terminal=cron res=success'
type=CRED_ACQ msg=audit(10/19/2015 15:50:01.877:15324) : pid=11620 uid=root auid=unset ses=unset subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_unix acct=root exe=/usr/sbin/crond hostname=? addr=? terminal=cron res=success'
type=LOGIN msg=audit(10/19/2015 15:50:01.878:15325) : pid=11620 uid=root subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old-auid=4294967295 auid=root old-ses=4294967295 ses=2076 res=yes
type=USER_START msg=audit(10/19/2015 15:50:01.889:15326) : pid=11620 uid=root auid=root ses=2076 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_loginuid,pam_keyinit,pam_limits,pam_systemd acct=root exe=/usr/sbin/crond hostname=? addr=? terminal=cron res=success'
type=CRED_REFR msg=audit(10/19/2015 15:50:01.889:15327) : pid=11620 uid=root auid=root ses=2076 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_unix acct=root exe=/usr/sbin/crond hostname=? addr=? terminal=cron res=success'
type=CRED_DISP msg=audit(10/19/2015 15:50:01.901:15328) : pid=11620 uid=root auid=root ses=2076 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_unix acct=root exe=/usr/sbin/crond hostname=? addr=? terminal=cron res=success'
type=USER_END msg=audit(10/19/2015 15:50:01.903:15329) : pid=11620 uid=root auid=root ses=2076 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_loginuid,pam_keyinit,pam_limits,pam_systemd acct=root exe=/usr/sbin/crond hostname=? addr=? terminal=cron res=success'

Sometimes, just the first line is shown and sometimes, the whole of the log is shown. When I log in and run the script as root, I get the issue, but when I login as splunk and run the script, I get nothing.

I checked the scripts and made sure that the WHOLE of the /opt/splunk directory is owned by splunk:splunk with 755 (as I thought that it was a permissions issue).

What am I doing wrong and is there something else that I need to install to get it working? There seem to be other, similar errors around.

I am running CentOS 7.0 with Splunk 6.2.3.

Kindest regards,

BlueSocket

elvisior
Explorer

This is how (I think) I fixed this error:

Change this line in rlog.sh from:
if [ -n "service auditd status" -a "$?" -eq 0 ] ; then
To:
if [ -n "service auditd status 2> /dev/null" -a "$?" -eq 0 ] ; then

Why this works?

Because on rhel7 the redirecting to systemctl comment is sent to stderr which splunk interprets as an error it should log under the ExecProcessor in splunkd.log

mpham
Engager

Thanks for posting this - this worked perfectly for me.

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

when in doubt update... there have been many fixes, including some to rlog.sh. http://docs.splunk.com/Documentation/UnixAddOn/5.2.0/User/Releasenotes

0 Karma

BlueSocket
Communicator

I think that I have worked out what is causing the error - the script is executing and the "service auditd status" line is causing the following, because bash on CentoOS 7 reinterpreting the command and redirecting it to the following and the command returns an error:

Redirecting to /bin/systemctl status  auditd.service

I think that the error value that is being returned is not REALLY an error, but a Warning, instead.

I see that the version of Splunk_TA_nix is 5.1.2 and that there is a revision (5.2.0) out. Will that fix this issue?

0 Karma

acain_splunk
Splunk Employee
Splunk Employee

we are seeing the same issue on redhat ver 3.10. it looks like the rlog.sh script is not expecting the os to reroute the service call to a 'different' service, so it writes an error. i have contacted the team responsible for development on the unix TA and opened a bug with them.

0 Karma

BlueSocket
Communicator

OK, wow! It looks like I have a genuine bug and it is not my mistake!

It might be an idea if we talk direct, as this is just one of many, similar, errors from the Splunk_TA_nix App, I think.

0 Karma

acain_splunk
Splunk Employee
Splunk Employee

any progress on this? I'm experiencing the same thing here.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...