Knowledge Management

If a file is ignored due to IgnoreOlderThan, is there any way to configure Splunk to read it if I need it later?

Flynt
Splunk Employee
Splunk Employee

If I specify a value for ignoreOlderThan for an input in my inputs.conf and tailingprocessor thinks the file is older than the ignoreOlderThan value and therefore ignores the file,
is there any way for the file to be read again by Splunk?

1 Solution

bmacias84
Champion

The answer is yes, but of course this depends on your requirements. The simiplist answer is to use oneshot using the CLI. You can issue one shot remotly on any splunk instance so long as you have enable remote access or have change the default password for the admin account.

I have on occasion scripted a remote oneshot to index files for one off cases.

If your looking to reset your fishbucket btprob is the real answer.

To reset one file in your fishbucket from your UF:

./btprobe -d SPLUNK_HOME/var/lib/splunk/fishbucket/splunk_private_db --file /var/log/access.log --reset

Note: Your splunk daemon or service must be stopped and if you try to use the base directory you will recieve "could not open '/var/log/' (Access is denied.)". If you doing this on a UF the btprobe will need to be copied to your UF.

You should recieve the following message: key=0x7b100f52c71e67f1 scrc=0x6e16f6c553b73581 sptr=8070178 fcrc=0xcdcd2890474a56fb flen=0 mdtm=1347001048 wrtm=1352329761**95Record (key 0x7b100f52c71e67f1) reset.

If in my case I have hundreds of test file sitting on a UF in mulitple directories, so I wrote a perl script to feed each file in each directory to the btprobe utility.

Hope this helps you or gets you started. Dont forget to accept or thumbs up answers.

Additional Reading:

Hope this help or gets you started.

View solution in original post

bmacias84
Champion

The answer is yes, but of course this depends on your requirements. The simiplist answer is to use oneshot using the CLI. You can issue one shot remotly on any splunk instance so long as you have enable remote access or have change the default password for the admin account.

I have on occasion scripted a remote oneshot to index files for one off cases.

If your looking to reset your fishbucket btprob is the real answer.

To reset one file in your fishbucket from your UF:

./btprobe -d SPLUNK_HOME/var/lib/splunk/fishbucket/splunk_private_db --file /var/log/access.log --reset

Note: Your splunk daemon or service must be stopped and if you try to use the base directory you will recieve "could not open '/var/log/' (Access is denied.)". If you doing this on a UF the btprobe will need to be copied to your UF.

You should recieve the following message: key=0x7b100f52c71e67f1 scrc=0x6e16f6c553b73581 sptr=8070178 fcrc=0xcdcd2890474a56fb flen=0 mdtm=1347001048 wrtm=1352329761**95Record (key 0x7b100f52c71e67f1) reset.

If in my case I have hundreds of test file sitting on a UF in mulitple directories, so I wrote a perl script to feed each file in each directory to the btprobe utility.

Hope this helps you or gets you started. Dont forget to accept or thumbs up answers.

Additional Reading:

Hope this help or gets you started.

DerekB
Splunk Employee
Splunk Employee

Tailing processor looks at the modtime of the file, so if you update the modtime, the file should be looked at again by Splunk. One easy way to do this is to simply restart the machine. The modtime gets updated during that process and tailing processor will then re-read the file and index any new information.

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...