I have a log source which re-uses the same log file based upon the day of the month. A filename contains other information, but that information doesn't change. Here is a sample: 10_40809_(IRL_Lab_Live).log
. That filename represents log data written on the 10th of October. Next month, it will be truncated, starting again from an empty file, but still the same filename. I've got a standard [monitor:...]
style inputs.conf stanza monitoring the directory containing these files.
It seems that I have to occasionally jiggle Splunk's elbow (restart) to get it to index "today's" data. Is there some other trick to get Splunk to index this smoothly? It doesn't sound like a candidate for alwaysOpenFile
, but I could try that setting if it would help.
I discovered that if we removed the ignoreOlderThan filter on the inputs and just ate the bullet on indexing all of that data that it will work fine. For whatever reason I guess that the filter was looking at the original file time?
I discovered that if we removed the ignoreOlderThan filter on the inputs and just ate the bullet on indexing all of that data that it will work fine. For whatever reason I guess that the filter was looking at the original file time?
Possible. Windows log sources are ... notorious for not updating the modtime of the file. It's seen a lot in IIS installations.