Getting Data In

Importing CSV

edgarsilva01
Path Finder

Hi everyone,

I am ingesting csv files that contain information about views of certain web pages,
These files are updated once a day.

I need that when the file is updated take only the new events.

Example: If the first file ends on day 20, for the second time it is updated I am no longer interested in seeing old events
I just want splunk to take the values ​​from day 21.

Will they have any idea how I can do it?

Thank you

Tags (1)
0 Karma

codebuilder
Influencer

Configure your forwarder to use "batch" instead of "monitor", and set "move_policy =sinkhole".

This will cause the forwarder to delete the file after it's been ingested. Once your process creates a new file, the forwarder will pick it up and you'll get only the new results.

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#BATCH_.28.22Upload_a_file.22_in...

Ignore the "Upload a File" portion in the documentation. Not sure why that is there, it's misleading.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Is the entire file re-written every day or are the latest results appended to the file?
What are the inputs.conf settings for the file?

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...