I see messages in the splunkd.log file:
09-07-2012 14:39:22.832 +0200 INFO BatchReader - Removed from queue file='/misc/tact/users/mk01232/agw/l1-iamprdagw25/reverse/MCS_RNL_2/http-reverse/extended/120727-A.log'.
Why is it removed from the queue? The file is asci, readable, correct permissions
Marc
Marc,
Removed from queue typically means that Splunk has reached an EOF and is finished reading the file.
Thanks,
--adam
I had the same issue for a csv log file with a header. My fix was to add the following line to my monitor rule in inputs.conf on the Universal Forwarder.
crcSalt= <SOURCE>
http://docs.splunk.com/Documentation/Splunk/latest/admin/inputsconf
crcSalt = <string>
* Use this setting to force Splunk to consume files that have matching CRCs (cyclic redundancy checks). (Splunk only
performs CRC checks against the first few lines of a file. This behavior prevents Splunk from indexing the same
file twice, even though you may have renamed it -- as, for example, with rolling log files. However, because the
CRC is based on only the first few lines of the file, it is possible for legitimately different files to have
matching CRCs, particularly if they have identical headers.)
* If set, <string> is added to the CRC.
* If set to the literal string <SOURCE> (including the angle brackets), the full directory path to the source file
is added to the CRC. This ensures that each file being monitored has a unique CRC. When crcSalt is invoked,
it is usually set to <SOURCE>.
* Be cautious about using this attribute with rolling log files; it could lead to the log file being re-indexed
after it has rolled.
* Defaults to empty.
I am having the same problem with a log file (not syslog) that is continually written too. However Splunk (on a UF) decides its done reading and stops sending the log file. What do I need to do to correct this?
Marc,
Removed from queue typically means that Splunk has reached an EOF and is finished reading the file.
Thanks,
--adam