Getting Data In

Any known method to alter polling interval for file monitoring?

dstaulcu
Builder

There is a log file I want want monitor with splunk universal forwarder (on windows) which receives line writes only during a phase of windows startup before the splunk universal forwarder is running..

I am able to successfully search line changes when subscribing to file-based inputs using the monitor input type. My only concern with this approach is that it appears that Splunk polls the file system for changes to file every second, which seems like wasted disk i/o.

I tried the MonitorNoHandle input type but that does not seem to work, I assume because the forwarder is not running at point in time when line changes occur.

Is there a way to control the interval of polling for file changes with the monitor input type? Is there an alternate method (short of script-based input) you would recommend? I can also just "get over" the wasted i/o line of thought. 🙂

0 Karma

dwaddle
SplunkTrust
SplunkTrust

How long did you watch it with procmon? The forwarder is supposed to back off and poll files less frequently when the files themselves are not being frequently updated. If you find this is not true, it may be worth a support case to find out why.

But, one thing to think about is that the OS will do a good job of caching these files. A once a second poll may seem like a lot, but it really doesn't add an appreciable amount of I/O to your system unless we are talking about millions of files. The kernel will keep this file in cache, and you won't do much actual I/O against it.

0 Karma

dstaulcu
Builder

Hello

I didn’t realize it would back-off over time. That would be a good option. So far I am 30+ minutes in from last file write but splunkd is still polling on 1s intervals. I will let this run for 24h and report back if the polling rate changes.

This type of file I/O is not a big deal to me for operating system instances having exclusive access to disk. My reservations come with use among virtualized systems sharing disk.

I'll take a lower level look at this to see what the implications to disk are... cause..learning and stuff.

Thanks!

0 Karma

dstaulcu
Builder

24h later. splunkd still polling on 1s intervals.

0 Karma

ppablo
Retired

Hi @dstaulcu

I was digging through documentation and saw this interval attribute that "If you set the interval to a negative number, Splunk Enterprise runs the input one time."
http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/MonitorWindowshostinformation#Windows_host_mo...

This is under the WinHostMon stanza, so I'm not sure if it'll apply to the monitor stanza input type, but it might be a setting worth testing out? When I checked http://docs.splunk.com/Documentation/Splunk/6.2.1/admin/inputsconf for the interval setting under WinHostMon, it wasn't as detailed as the table included in the first docs link, so maybe there's hope with testing that setting. When it says "runs the input one time", I'm also not sure if that means one time for all-time, or each time Splunk Enterprise (UF) and/or Windows machine is restarted. Otherwise, seems like the interval can just be set to another number so it doesn't poll every second.

0 Karma

dstaulcu
Builder

Thanks for digging and clever idea! I just gave that a shot but no dice!

Despite the change, sysinternals procmon still exposes persistent polling of the file by splunkd every second.

0 Karma

dstaulcu
Builder

follow up - feature modification request submitted.

0 Karma

Jedi
Observer

Any update? 

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...