For testing purposes, I would really really like to force splunk to poll files in a monitor:// directory structure (and index changes) more quickly - at least temporarily. Is there some way I can force splunk to do this, overriding the default check-backoff behavior?
"Generally speaking, I haven't seen the need for this though, as I see files indexed with almost no delay at all between when the files are updated and when the updates come into Splunk."
My experiments show that there's anywhere from a zero to 15 second delay in indexing, which seems to match up well with the answer to this post:
http://splunk-base.splunk.com/answers/6693/how-to-force-splunk-to-index-new-files-quickly
I've since found a solution for what I want to happen - use oneshot:
which converts indexing from async to blocking - more appropriate for my usage scenario.
There isn't a user configurable monitor interval setting which can be used to gain this kind of control over monitor stanzas. Generally speaking, I haven't seen the need for this though, as I see files indexed with almost no delay at all between when the files are updated and when the updates come into Splunk.
The only time I've seen this an issue with data taking too long to get there was with previous versions and trying to monitor a directory with too many files/subdirectories. Is your monitor stanza set overly broad then trying to whitelist down?