Hi,
I'm faced with a weird issue where the /opt/splunk/etc/apps/Splunk_TA_nix/bin/rlog.sh does not do anything. I'm using Amazon Linux.
I ran an echo $KERNEL on the command prompt and there was nothing returned, hence the script code (below copied from the rlog.sh file) doesn't proceed further to where it will read the /var/log/audit/audit.log. I have confirmed that my rlog.sh has execute permissions by the splunk userid and the audit.log also has read ACL.
Shall I edit the script to just execute anyway by removing the check for the kernel type ? In that case, does "assertInvokerIsSuperuser" will also need to be removed.
SEEK_FILE=$SPLUNK_HOME/var/run/splunk/unix_audit_seekfile
AUDIT_FILE=/var/log/audit/audit.log
if [ "x$KERNEL" = "xLinux" ] ; then
assertInvokerIsSuperuser
assertHaveCommand service
assertHaveCommandGivenPath /sbin/ausearch
if [ -n "`service auditd status`" -a "$?" -eq 0 ] ; then
if [ -e $SEEK_FILE ] ; then
SEEK=`head -1 $SEEK_FILE`
else
SEEK=0
echo "0" > $SEEK_FILE
fi
FILE_LINES=`wc -l $AUDIT_FILE | cut -d " " -f 1`
if [ $FILE_LINES -lt $SEEK ] ; then
# audit file has wrapped
SEEK=0
fi
awk -v START=$SEEK -v OUTPUT=$SEEK_FILE 'NR>START { print } END { print NR > OUTPUT }' $AUDIT_FILE | tee $TEE_DEST | /sbin/ausearch -i 2>/dev/null | grep -v "^----"
fi
elif [ "x$KERNEL" = "xSunOS" ] ; then
:
elif [ "x$KERNEL" = "xDarwin" ] ; then
:
elif [ "x$KERNEL" = "xHP-UX" ] ; then
:
elif [ "x$KERNEL" = "xFreeBSD" ] ; then
:
fi
Any hint/direction is highly appreciated!
And for assertHaveCommand() the following needs to be updated (line 47 of bin/common.sh from Splunk_TA_nix 6.0.2):
# # # /sbin/ is often absent in non-root users' PATH, and we want it for ifconfig(8)
PATH=$PATH:/sbin/
Ubuntu needs this, or it wont be able to find the "service" command when Splunk is running as non-root (splunk).
# # # Append path to help find commands when running as non-root, as the non-root paths are different
PATH=$PATH:/sbin/:/usr/sbin/
It's called from bin/rlog.sh as follows:
assertHaveCommand service
Cheers,
Chris.
An update on this after a little digging..
In Splunk_TA_nix 6.0.2, It looks like the rlog.sh script is intended to run as root (the implication is Splunk runs as root), per the following check in common.sh:
assertInvokerIsSuperuser ()
{
[ `id -u` -eq 0 ] && return
echo "Must be superuser to run this script, quitting" > $TEE_DEST
exit 1
}
If you enable debugging on rlog.sh (looks like it throws away this important output to /dev/null inside $TEE_DEST unless you have debug enabled):
sudo su - splunk
$SPLUNK_HOME/bin/splunk cmd $SPLUNK_HOME/etc/apps/Splunk_TA_nix/bin/rlog.sh --debug
You'll notice a debug file named debug--rlog.sh-- in the cwd which (unsurprisingly) says:
Must be superuser to run this script, quitting
Per the logic in assertInvokerIsSuperuser()
From an ES point-of-view, this is sub-optimal not only from a security standpoint (running Splunk as root), but the TA is of course designed to work from sourcetype=auditd keyed from the rlog.sh input, so it's not adding the value it could.
It's designed this way (i'd guess) because /var/log/audit/audit.log by way of ausearch is out-of-the-box only visible by root (without changes), but with proper unix/posix permissions setup, Splunk running as splunk, can ingest the file via ausearch, rlog.sh, etc.
It's debatable whether it's a security risk allowing a non-root user to read the audit.log file, but if you can't bring it up into Splunk to keep eyes on it, it's a relatively small risk to accept.
Anyway, just wanted to get to the bottom of why that was happening.. 🙂
PS: There's also a bug in assertHaveCommand() (at least on Ubuntu) i had to also work-around after assertInvokerIsSuperuser() to get it to work but i haven't yet found the root-cause for that, just a work-around, but looking into it..
Cheers,
Chris.
The KERNEL variable is not set within the OS but rather from another script within the Splunk_TA_nix. As the first line of the rlog.sh consists of:
. `dirname $0`/common.sh
Within this common.sh the KERNEL variable is set using:
# # # what OS is this?
KERNEL=`uname -s`
This is common across all Splunk_TA_nix scripts in how they run as many things are OS dependent. I would ensure that this common.sh is running properly as the need for commenting out the KERNEL if/elif segments shouldn't be needed.
I got it working by commenting out the following lines. Apparently Amazon Linux has all the dependencies satisfied as any other mainstream Linux, however the $KERNEL variable is not set (or may be need to be set when creating the EC2 instance, perhaps ?).
#if [ "x$KERNEL" = "xLinux" ] ; then
# assertInvokerIsSuperuser
# assertHaveCommand service
# assertHaveCommandGivenPath /sbin/ausearch
if [ -n "`service auditd status`" -a "$?" -eq 0 ] ; then
if [ -e $SEEK_FILE ] ; then
SEEK=`head -1 $SEEK_FILE`
else
SEEK=0
echo "0" > $SEEK_FILE
fi
FILE_LINES=`wc -l $AUDIT_FILE | cut -d " " -f 1`
if [ $FILE_LINES -lt $SEEK ] ; then
# audit file has wrapped
SEEK=0
fi
awk -v START=$SEEK -v OUTPUT=$SEEK_FILE 'NR>START { print } END { print NR > OUTPUT }' $AUDIT_FILE | tee $TEE_DEST | /sbin/ausearch -i 2>/dev/null | grep -v "^----"
fi
# elif [ "x$KERNEL" = "xSunOS" ] ; then
# :
# elif [ "x$KERNEL" = "xDarwin" ] ; then
# :
# elif [ "x$KERNEL" = "xHP-UX" ] ; then
# :
# elif [ "x$KERNEL" = "xFreeBSD" ] ; then
# :
# fi