Getting Data In

About data input, some data didn't be eaten.

flora123
Path Finder

Hi dears,

I have a problem about the data input.

I monitored a directory, and found some data didn't be eaten. I don't know what's wrong with it.

My server works on Linux.

I try to move these file to Windows, and use the same props.conf.

Strange thing happened! I can find the data that they can't be searched on the Linux server.

I clean the index many times, wait several hours, but all useless.

Some people encountered the same situation?

Thanks a lot. 😃

Tags (1)
0 Karma
1 Solution

justinhart
Path Finder

Add:

crcSalt = <SOURCE>

as in:

[monitor://xxxxxxxxxxxxxxx]
disabled = 0
followTail = 0
host = xxxxxxxxxxxxxxx
index = xxxxxxxxxxxxxxx
crcSalt = <SOURCE>
sourcetype = iis_w3c_default

to your input in inputs.conf.

This should be typed exactly and splunk will have to be restarted. Monitor the logs again to see if you keep getting the errors you mentioned. Also watch for your data to start appearing. Let me know how it goes


Please check your splunkd.log file for errors related to the files you are trying to monitor with the following search command:

index="_internal" " error " NOT debug source="*splunkd.log*"

You can specify a time range to narrow your results.

Also, is the directory you are trying to monitor on windows or linux. And I believe that your index server is linux, is that correct?

View solution in original post

hulahoop
Splunk Employee
Splunk Employee

On Linux, are you running Splunk as root or another user? If running as a different user, you might want to check the user has permissions to access all files in the directory you are monitoring.

0 Karma

flora123
Path Finder

I used 'chmod 777 '.But look no effect...So I change owner of the file to splunk.I used 'chown splunk:splunk '.These data still don't be eaten.I don't know what should I do...

0 Karma

flora123
Path Finder

Thanks, hulahoop. I login as root, and decompress these files to a folder.I will try to change these permissions of files to '0777'.But I am a bit confused, why some data in the file be not eaten? If the problem is the permissions, should all the data in the file be not eaten? Thanks. 😃

0 Karma

justinhart
Path Finder

Add:

crcSalt = <SOURCE>

as in:

[monitor://xxxxxxxxxxxxxxx]
disabled = 0
followTail = 0
host = xxxxxxxxxxxxxxx
index = xxxxxxxxxxxxxxx
crcSalt = <SOURCE>
sourcetype = iis_w3c_default

to your input in inputs.conf.

This should be typed exactly and splunk will have to be restarted. Monitor the logs again to see if you keep getting the errors you mentioned. Also watch for your data to start appearing. Let me know how it goes


Please check your splunkd.log file for errors related to the files you are trying to monitor with the following search command:

index="_internal" " error " NOT debug source="*splunkd.log*"

You can specify a time range to narrow your results.

Also, is the directory you are trying to monitor on windows or linux. And I believe that your index server is linux, is that correct?

flora123
Path Finder

Great!Thank you very much! It works! 😃

0 Karma

justinhart
Path Finder

Sorry about the above comment didn't show correctly. Please see my initial answer for the revisions.

0 Karma

justinhart
Path Finder

Add:

crcSalt =

as in:

[monitor://xxxxxxxxxxxxxxxxxx]
disabled = 0
followTail = 0
host = xxxxxxxxxxxx
index = xxxxxxxxx
crcSalt =
sourcetype = iis_w3c_default

to your input in inputs.conf. This should be typed exactly and splunk will have to be restarted. Monitor the logs again to see if you keep getting the errors you mentioned. Also watch for your data to start appearing. Let me know how it goes.

0 Karma

flora123
Path Finder

Thanks, justinhart.I find many errors about 'TailingProcessor - Ignoring path due to: File will not be read, is too small to match seekptr checksum...'.I think it may be about the permissions. I will try and tell you. And I don't setup a index server of Splunk.I just put them on one computer.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...