- Apps and Add-ons
- :
- All Apps and Add-ons
- :
- Will ignoreOlderThan permanently ignore a file or ...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
From the inputs.conf.spec ignoreOlderThan
Causes the monitored input to stop checking files for updates if their
modtime has passed this threshold.
We are monitoring Oracls DB audit trail files. The applications generates a separate file for each session. This can easily lead to thousands of files being created every hour. As such we need to set our ignoreOlderThan threshold very low (4h) to keep performance reasonable.
For 99% of these logs that is not a problem, but it's quite possible to have sessions that have a new entry appended after several hours. So the modtime will be updated. The last Answer I see on this topic (https://answers.splunk.com/answers/151149/does-splunk-re-index-a-file-that-was-ignored-due-t.html#co...) suggests that even though the modtime will change, if a file ever fell out of the ignoreOlderThan threshold it will NOT be checked unless the forwarder restarts.
Can anyone confirm if this is still the case in 6.3 + ?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As a work-around, maybe you can increase the ignoreOlderThan by a day or so and exclude this day of data at the indexer level. We do pay, in such a case, for the license traffic for the extra day.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
For reference, we saw the forwarder memory usage spike at about 8GB when ignoreOlderThan was more than 4 hours.
Not Splunks fault, its just the way oracle writes its files out.
Also Batch mode is a non starter, as Oracle will not recreate audit session files after they are deleted.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The behavior has not changed in 6.3/6.4.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yeah that's what we saw from testing.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
haha at first I only saw the title of your question and was about to share a post from 2 years ago related to this topic, but read through your entire explanation and saw you already referenced it *whistles and walks away...runs back* but I do hope you do get confirmation whether or not this forwarder behavior has changed 🙂 interesting topic!
Cheers!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
lol - plz my splunk answers fu is strong 🙂
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
any answer ?
Introducing the 2024 SplunkTrust!
Introducing the 2024 Splunk MVPs!
Splunk Custom Visualizations App End of Life
