Getting Data In

How to avoid duplicated event

hochit
Path Finder

We have csv type of data file which is overwritten and with new data appended to the end every night. I found Splunk load/duplicate all the data again everyday!

As I know crcSalt only check CRC with first few lines of the file. How Splunk works in this case to identify only end of the file has new data?

followTail works for file replace?

Tags (3)
0 Karma

barne_dn
Explorer

How are you loading the file:

Are you using a Splunk monitor?
http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/Monitorfilesanddirectories

That might be your problem.

0 Karma

dlovett
Path Finder

I am having the exact same issue! did you figure out a solution?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...