Splunk Search

How to move uploaded file to directory splunk is monitoring?

Takajian
Builder

My splunk instance monitored the directory where proxy server upload compressed access log to via ftp. However my splunk instance sometimes indexed event twice, it result in duplicate events.

I got answer by splunk engineer that Splunk tries hard to read uncompleted file. Sometimes it fails to read. Other time splunk might be able to read. Upload the file to one directory where Splunk is not monitoring, and move it to the directory splunk is monitoring.

My question is if anybody have experience to move the uploaded file to directory splunk is monitoring, please share your experience with me. I think splunk can not do it, I will need to achieve it by using os command or script. I would like to know which os command or what script you used and move the file safely.

Tags (1)
0 Karma

dwaddle
SplunkTrust
SplunkTrust

Check this post for information related to Splunk and atomic operations. http://answers.splunk.com/questions/6482/appending-vs-overwriting-tailed-log-files

This is something you will have to implement outside of Splunk proper, but is manageable as long as you make all of your operations filesystem-atomic

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...