index a config file on a regular basis, even if not changed

[mataharry]

I want to monitor configuration files, to keep track of the config everyday.
I don't care that the file changed or not, I just want a copy with the daily version.

How to setup this type of inputs ?

[yannK]

Here is an example of inputs + script for windows and linux.

/myapp/local/inputs.conf
remark : this scripted input does not send the data, it just triggers the oneshot CLI comnand.
Add the path and the filter as arguments,
Cron notation for the interval, once a day at 1am


[script://.\bin\oneshot_folder_windows.bat c:\mypath\ *.txt]
disabled = true
index = main
source = oneshot
sourcetype = oneshot
interval = 0 1 * * *

[script://./bin/oneshot_folder_linux.sh /mypath *.txt]
disabled = false
index = main
source = oneshot
sourcetype = oneshot
interval = 0 1 * * *

/myapp/bin/oneshotfolderlinux.sh

#!/bin/bash
# linux script to index oneshot recursively the files from a folder
foldertomonitor=$1
filter=$2
file_list=find $foldertomonitor -name $filter
for file in $file_list
do
$SPLUNK_HOME/bin/splunk add oneshot $file -auth admin:changeme
done


/myapp/bin/oneshotfolderwindows.bat

echo off
rem bat script to index oneshot all the txt files from a folder
set foldertomonitor=%1
set filter=%2
set splunkexec=%SPLUNK_HOME%\bin\splunk.exe
rem update the paths for the temp file
set tempfilelist=c:\tmp\filelist.txt
mkdir c:\tmp
del %tempfilelist%
forfiles /s /p %foldertomonitor% /M %filter% /c " cmd /c echo @path >> %tempfilelist% "
for /F "delims=" %%G in (%tempfilelist%) do "%splunkexec%" add oneshot %%G -auth admin:changeme



[yannK]

terrible html parsing if messing the code, beware.

[yannK · Accepted solution]

This is not the intended behavior of splunk monitoring, that will not reindex a file until it changed.

Here is one method to achieve this :

• use the CLI command "add oneshot //path/to/my/file" to index the file at will.
  • see http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/MonitorfilesanddirectoriesusingtheCLI
• setup the command in a scripted input, and define the scripted input interval with a cronjob notation.
  • see http://docs.splunk.com/Documentation/Splunk/4.3.1/Developer/ScriptedInputsIntro