Events missing on audit.d log rollover with rlog.sh

[LeeSart]

This is a solution post rather than a question. I'm posting in the hope that it is relevant to other users of rlog.sh.

My company is using the Splunk_TA_nix add on and rlog.sh to ingest audit.d logs into Splunk. As these are used for auditing purposes we have to ensure there are no missing events. I noticed that we were missing a few (1-10) events each time the audit.d log rolled over. This is due to the way rlog.sh is written - if the log rolls over the script just resets the checkpoint value to 0 and starts reading from the top of the new log.

I've altered rlog.sh so when the log rolls over, the remainder of the old log file is read before resetting the checkpoint to 0. This ensures that no events are missed on log rollover. Here's the altered script:

#!/bin/sh
# Copyright (C) 2005-2015 Splunk Inc. All Rights Reserved.
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.
#
# credit for improvement to http://splunk-base.splunk.com/answers/41391/rlogsh-using-too-much-cpu
. `dirname $0`/common.sh

SEEK_FILE=$SPLUNK_HOME/var/run/splunk/unix_audit_seekfile
AUDIT_FILE=/var/log/audit/audit.log
AUDIT_FILE_PREV=/var/log/audit/audit.log.1

if [ "x$KERNEL" = "xLinux" ] ; then
#    assertInvokerIsSuperuser
    assertHaveCommand service
    assertHaveCommandGivenPath /sbin/ausearch
    if [ -n "`service auditd status`" -a "$?" -eq 0 ] ; then
            if [ -a $SEEK_FILE ] ; then
                SEEK=`head -1 $SEEK_FILE`
            else
                SEEK=0
                echo "0" > $SEEK_FILE
            fi
            FILE_LINES=`wc -l $AUDIT_FILE  | cut -d " " -f 1`
            if [ $FILE_LINES -lt $SEEK ] ; then
                # audit file has wrapped

                # create Seek_Prev to get final events from rolled file
                SEEK_PREV=`echo $SEEK`

                # Set Seek to 0 to read from start of new file
                SEEK=0
            else
                # Set SEEK_PREV to a large enough number that the first awk statement returns nothing
                SEEK_PREV=10000000000000
            fi

           # get final events from rolled file (if rolled)
            awk -v START=$SEEK_PREV -v OUTPUT=$TEE_DEST 'NR>START { print } END { print NR > OUTPUT }' $AUDIT_FILE_PREV | tee $TEE_DEST | /sbin/ausearch -i 2>/dev/null | grep -v "^----"

            awk -v START=$SEEK -v OUTPUT=$SEEK_FILE 'NR>START { print } END { print NR > OUTPUT }' $AUDIT_FILE | tee $TEE_DEST | /sbin/ausearch -i 2>/dev/null | grep -v "^----"
    fi
elif [ "x$KERNEL" = "xSunOS" ] ; then
    :
elif [ "x$KERNEL" = "xDarwin" ] ; then
    :
elif [ "x$KERNEL" = "xHP-UX" ] ; then
        :
elif [ "x$KERNEL" = "xFreeBSD" ] ; then
        :
fi

[youngsuh]

For those of you who run has splunk has service.  You need to comment below.  (for unix newbies)

#    assertInvokerIsSuperuser  

[martin_mueller · Accepted solution]

Question answers itself 🙂