From the inputs.conf.spec ignoreOlderThan
Causes the monitored input to stop checking files for updates if their
modtime has passed this threshold.
We are monitoring Oracls DB audit trail files. The applications generates a separate file for each session. This can easily lead to thousands of files being created every hour. As such we need to set our ignoreOlderThan threshold very low (4h) to keep performance reasonable.
For 99% of these logs that is not a problem, but it's quite possible to have sessions that have a new entry appended after several hours. So the modtime will be updated. The last Answer I see on this topic (https://answers.splunk.com/answers/151149/does-splunk-re-index-a-file-that-was-ignored-due-t.html#co...) suggests that even though the modtime will change, if a file ever fell out of the ignoreOlderThan threshold it will NOT be checked unless the forwarder restarts.
Can anyone confirm if this is still the case in 6.3 + ?
As a work-around, maybe you can increase the ignoreOlderThan
by a day or so and exclude this day of data at the indexer level. We do pay, in such a case, for the license traffic for the extra day.
For reference, we saw the forwarder memory usage spike at about 8GB when ignoreOlderThan
was more than 4 hours.
Not Splunks fault, its just the way oracle writes its files out.
Also Batch mode is a non starter, as Oracle will not recreate audit session files after they are deleted.
The behavior has not changed in 6.3/6.4.
Yeah that's what we saw from testing.
haha at first I only saw the title of your question and was about to share a post from 2 years ago related to this topic, but read through your entire explanation and saw you already referenced it *whistles and walks away...runs back*
but I do hope you do get confirmation whether or not this forwarder behavior has changed 🙂 interesting topic!
Cheers!
lol - plz my splunk answers fu is strong 🙂
any answer ?