Getting Data In

Why isn't the data in the file indexed at all?

manus
Communicator

It's a simple Splunk install, just on one server.
On the file & directories data inputs screen, I have set up a continous file input for a given folder.
This input worked the first time: I copied a file in the folder, and it got indexed as expected.
Now I'm copying a file to it again, I can see the "number of files" column incrementing, so it looks like Splunk saw the new file, but the data is not indexed.
Anybody has an idea about what i can do?

Tags (2)
0 Karma
1 Solution

manus
Communicator

Thanks to somesoni, I ran:
index=_internal sourcetype=splunkd component=TailingProcessor '/data/scada/testlogs/capacity/CapacityIndex01-09-2014.txt'
which returned:
01-16-2015 11:19:10.811 +0000 WARN TailingProcessor - Insufficient permissions to read file='/data/scada/testlogs/capacity/CapacityIndex01-09-2014.txt' (hint: Permission denied).

I guess it has to do with Splunk process account not having sufficient priviledges to read in the folder where the file was written.
I created another input in another folder, and copied the same file there, and it worked, the data got indexed.

View solution in original post

0 Karma

manus
Communicator

Thanks to somesoni, I ran:
index=_internal sourcetype=splunkd component=TailingProcessor '/data/scada/testlogs/capacity/CapacityIndex01-09-2014.txt'
which returned:
01-16-2015 11:19:10.811 +0000 WARN TailingProcessor - Insufficient permissions to read file='/data/scada/testlogs/capacity/CapacityIndex01-09-2014.txt' (hint: Permission denied).

I guess it has to do with Splunk process account not having sufficient priviledges to read in the folder where the file was written.
I created another input in another folder, and copied the same file there, and it worked, the data got indexed.

0 Karma

somesoni2
Revered Legend

Run this query and check the events for reason.

index=_internal sourcetype=splunkd component=TailingProcessor "YourFileName"

manus
Communicator

Thanks very much somesoni, I get one line:

01-16-2015 11:19:10.811 +0000 WARN TailingProcessor - Insufficient permissions to read file='/data/scada/testlogs/capacity/CapacityIndex01-09-2014.txt' (hint: Permission denied).

0 Karma

kml_uvce
Builder

as you are trying same file again and again it will no indexed
use crcSalt in inputs.conf

http://docs.splunk.com/Documentation/Splunk/6.2.1/admin/inputsconf

0 Karma
Get Updates on the Splunk Community!

Splunk APM: New Product Features + Community Office Hours Recap!

Howdy Splunk Community! Over the past few months, we’ve had a lot going on in the world of Splunk Application ...

Index This | Forward, I’m heavy; backward, I’m not. What am I?

April 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

A Guide To Cloud Migration Success

As enterprises’ rapid expansion to the cloud continues, IT leaders are continuously looking for ways to focus ...