I have an app that is not removing/deleting the files after consuming them. They are indexed appropriately, but just not deleted afterwards.
inputs.conf
[batch:///opt/splunk/etc/apps/my-special-app/pickup/*.json]
index = test
sourcetype = nessus_json
move_policy = sinkhole
I have tested this on a second Splunk box and the exact same app will correctly remove the files after indexing them. I can't tell where the issue may be on this main Splunk box, however. Any suggestions?
On Splunk v6.2.1. This worked a month or so ago. I'd rather figure out the cause before moving to upgrade the Splunk instance.
I forced a permissions issue with a file such that the Splunk user had read, not write permissions to a file that was configured as a batch input. It resulted in this line in splunkd.log
:
11-24-2017 22:49:10.062 +0000 ERROR TailReader - Unable to remove sinkhole file: path=/tmp/batch_del_fail.log, errno=Operation not permitted
Can you look for a similar message to verify if it is a simple permissions issue?
Sounds like the permissions on the filesystem got screwed up. Check to make sure the account splunk is running as can actually delete the files.