Getting Data In

monitor a directory with yesterday date (and avoid monitoring directories with today date..)

mekamundia
Explorer

hi everybody

i have to monitor a server with some directories ordered by date. I have to index automatically every day the folder with yesterday date and I absolutely have to avoid monitoring the folders with today date: in those directories, the server instead append rows to files, again and again erases and rewrites the files, updating them, and forcing Splunk to re-indexing all events. I would tell Splunk to follow a command like this [monitor:///home/folder1/folder2/folder3//.../LOG.txt] but... i don't know how insert a date-variable in "monitor". How I can have a variable in monitor path (or how I can use a workaround to obtain my goal..)?
here a sample of my the directories

///home/folder1/folder2/folder3/2014-12-10/<some other directories and files>/LOG.txt
///home/folder1/folder2/folder3/2014-12-09/<some other directories and files>/LOG.txt
///home/folder1/folder2/folder3/2014-12-08/<some other directories and files>/LOG.txt
///home/folder1/folder2/folder3/2014-12-07/<some other directories and files>/LOG.txt

thanks in advance for help!

Tags (2)
0 Karma
1 Solution

changux
Builder

In my opinion, the best option is solve in linux (or the *NIX that you use):

1.) Create a script like this:

#!/bin/sh
rm -f /home/folder1/folder2/folder3/YESTERDAY
YESTERDAY=`date --date='yesterday' +%Y-%m-%d`
ln -s /home/folder1/folder2/folder3/$YESTERDAY /home/folder1/folder2/folder3/YESTERDAY

2.) Create an input to the YESTERDAY symbolic link:

[monitor:///home/folder1/folder2/folder3/YESTERDAY]
disabled = false
followTail = 0
sourcetype = any_sourcetype

3.) Create a cron job that execute the script at 00:01 every day.

1 0 * * * /path/to/script.sh

4.) Restart splunk

View solution in original post

0 Karma

mekamundia
Explorer

thanks a lot !

changux
Builder

In my opinion, the best option is solve in linux (or the *NIX that you use):

1.) Create a script like this:

#!/bin/sh
rm -f /home/folder1/folder2/folder3/YESTERDAY
YESTERDAY=`date --date='yesterday' +%Y-%m-%d`
ln -s /home/folder1/folder2/folder3/$YESTERDAY /home/folder1/folder2/folder3/YESTERDAY

2.) Create an input to the YESTERDAY symbolic link:

[monitor:///home/folder1/folder2/folder3/YESTERDAY]
disabled = false
followTail = 0
sourcetype = any_sourcetype

3.) Create a cron job that execute the script at 00:01 every day.

1 0 * * * /path/to/script.sh

4.) Restart splunk

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...