I have a couple of directories with 120,000 files in each. This is a file every 5 minutes or so for the last 6 months.
Will ignoreOlderThan make this faster to monitor, or simply listing these massive directories still take a long time?
There is only a file added every 5 minutes. Is there a setting that affects how often it will scan the directory?
ignoreOlderThan
will help, as (once it has listed the directory, which actually shouldn't take that long...even a mediocre network drive should be able to list out that number of files in less than 5 or 10 minutes) it doesn't even have to go back and check to see if they've been modified. On a local or fast drive, 120,000 files isn't really a problem to monitor anyway, but if your drive is on a network (likely) and slow, then it will help to ignore things past a certain age. The directory is basically checked as fast as Splunk can.