Hello,
We recently started to notice that a file that used to be monitored fine is no longer being pulled into splunk and nothing has changed (at least that Im aware of). Other files are being processed fine from the UF to the indexer but there is one specific file that can no longer be read, messages_processed.log. Below is the message that we are receiving with debug tracing on. I also tried adding crcSalt =
06-01-2013 17:10:44.452 -0400 DEBUG WatchedFile - seeking /usr/mr/shrm/log/messages_processed.log to off=21212158
06-01-2013 17:10:44.452 -0400 DEBUG WatchedFile - Reached EOF: fname=/usr/mr/shrm/log/messages_processed.log fishstate=key=0xbd17ad8dea12cfe6 sptr=21212158 scrc=0x5ae4ab6788b2c4f6 fnamecrc=0x8f47bfbec009b200 modtime=1370121040
06-01-2013 17:10:44.453 -0400 DEBUG TailingProcessor - Hit EOF immediately.
06-01-2013 17:10:44.453 -0400 DEBUG TailingProcessor - Have definitely hit EOF.
There is a similar question located here.
https://answers.splunk.com/answers/7568/splunk-not-indexing-some-files.html
They fixed the TIME_FORMAT for this input in props.conf and it seems to fixed it form them.
I had similar problems with some IIS logs, and had to emtpy that forwarders fishbucket before it would notice any change in the file. This will force the forwarder to reindex everything that is monitored by that forwarder.. So, use with caution as it may result in duplicate data. You might point the fowarder to a dev/test indexer for this one, or a throw-away index.
./splunk clean eventdata _thefishbucket