I've seen a few similar questions asked with answers that either don't apply or don't help, and I apologize in advance if I missed the helpful one somewhere. I'm fairly green on the forwarders so I may be missing something.
I've got the universal forwarder installed on a server and monitoring a single iis log location. I tracked down and am using the inputs.conf file in Program Files\SplunkUniversalForwarder\etc\apps\MSICreated\local. It currently has a single entry:
[monitor://M:\web_logs\site_directory*.log]
sourcetype = iis
ignoreOlderThan = 1d
followTail = 0
disabled = false
The server happens to live in eastern time, I'm in central and of course IIS logs in UTC. I added an entry in Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf under the [default] stanza "_tzhint = US/Central" so the logs are delivered in my time.
What I end up getting in the RealTime view is logs that are interpreted as "local" time...i.e. a log entry stamped as 2012-10-19 16:39:54 is indexed as 4:39 pm.
Also, I've noticed that the logs are consistently behind by between 3-8 minutes. That is, something logged on the server at say 10:39am doesn't show up in the index until 10:42 (though the index time is correct). At first I thought this was related to the IIS log buffering and flushing, but I can see log entries in the log file quite a while before they make it over.
I've checked the logs and routinely see entries like:
10-19-2012 12:45:17.605 -0400 WARN TcpOutputProc - Raw connection to ip=184.73.47.206:9997 timed out
10-19-2012 12:45:37.607 -0400 WARN TcpOutputProc - Raw connection to ip=184.73.47.206:9997 timed out
10-19-2012 12:45:40.123 -0400 INFO TailingProcessor - ...continuing.
10-19-2012 12:45:40.123 -0400 INFO BatchReader - Continuing...
10-19-2012 12:45:50.124 -0400 INFO BatchReader - Could not send data to output queue (parsingQueue), retrying...
10-19-2012 12:45:53.515 -0400 INFO TailingProcessor - Could not send data to output queue (parsingQueue), retrying...
10-19-2012 12:46:17.751 -0400 INFO TcpOutputProc - Connected to idx=67.202.7.237:9997 using ACK.
10-19-2012 12:47:13.831 -0400 INFO TcpOutputProc - Connected to idx=67.202.7.237:9997 using ACK.
But then I'll see some successes:
10-19-2012 12:37:17.997 -0400 INFO TcpOutputProc - Connected to idx=184.73.47.206:9997 using ACK.
Thanks in advance.
... View more