Hi
I am trying to index a file from different subdirectory but Splunk is not indexing some of those files for some weird reason. All subdirectories contain different files but I am just interested in a file with an specific extension (.tir) so I am using a whilelist (.tir$). Splunk indexed almost all files but there are some files that Splunk just didn't index. Do you know a reason why is this happening?
I examined the files and they are normal (Same type data, same type format, and same extension).
I found a solution to my own problem. I need to add:
crcSalt =
initCrcLength = 2000
I found a solution to my own problem. I need to add:
crcSalt =
initCrcLength = 2000
What exactly does the setting do ?
Essentially how many bytes in Splunk will check at the beginning of a file to try to uniquely identify it.
From: http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf
initCrcLength =
* How much of a file, in bytes, that the input reads before trying to
identify whether it is a file that has already been seen. You might want to
adjust this if you have many files with common headers (comment headers,
long CSV headers, etc) and recurring filenames.
* Cannot be less than 256 or more than 1048576.
* CAUTION: Improper use of this setting will cause data to be re-indexed. You
might want to consult with Splunk Support before adjusting this value - the
default is fine for most installations.
* Default: 256 (bytes).
Please use below code in monitor stanza
whiltelist = (*.tir)$
recursive = true
I am already indexing recursively but Splunk is not indexing for some subdirectories.
Can you please check on forwarder with below command whether all files with (.tlr) is showing or not in command output?
$SPLUNK_HOME/bin/splunk list monitor
I'm working with Windows OS.
I was examining the files and I noticed that there are very similar, almost the same. Is it possible that Splunk believed I am duplicating a file? If so, how could I change it so Splunk would index the file?
I didn't solve the problem. I checked the wrong index...
Anyway I tried to use CHECK_METHOD AND crcSalt but they didn't change anything.
But I found many errors in index=_internal which is weird because I tried using crcSalt = and the problem wasn't resolve. Looks like I have added these files previously which is wrong because I just created the index.
ERROR TailingProcessor - File will not be read, seekptr checksum did not match (file=C:\blah\blah\30343.tir). Last time we saw this initcrc, filename was different. You may wish to use a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.
Well in Splunk>Data Input> Files & Directory appears that there 4835 files which contain the .tir extension and they are supposed to be indexed.
Then use below command
$SPLUNK_HOME/bin/splunk.exe list monitor
For example:
C:\Program Files\SplunkUniversalForwarder\bin>splunk.exe list monitor
I checked the current size of the index (36MB) and the event count (1,854) which look normal.