Getting Data In

Can you auto remove CSV files after indexing?

jiaqya
Builder

Is there a configuration in Splunk where it can remove/move a CSV file after it has been indexed? so it does not show as active in the configured data input folder for being scanned as a valid file..( since its already indexed )

John.

0 Karma
1 Solution

MuS
SplunkTrust
SplunkTrust

Hi jiaqya,

you are looking for the [batch://...] stanza in inputs.conf, here are the docs http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#BATCH_.28.22Upload_a_file.22_in_...

[batch://<path>]
* A one-time, destructive input of files in <path>.
* For continuous, non-destructive inputs of files, use 'monitor' instead.

# Additional settings:

move_policy = sinkhole
* IMPORTANT: This setting is required. You *must* include
  "move_policy = sinkhole" when you define batch inputs.
* This setting causes the input to load the file destructively.

Hope this helps ...

cheers, MuS

View solution in original post

MuS
SplunkTrust
SplunkTrust

Hi jiaqya,

you are looking for the [batch://...] stanza in inputs.conf, here are the docs http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#BATCH_.28.22Upload_a_file.22_in_...

[batch://<path>]
* A one-time, destructive input of files in <path>.
* For continuous, non-destructive inputs of files, use 'monitor' instead.

# Additional settings:

move_policy = sinkhole
* IMPORTANT: This setting is required. You *must* include
  "move_policy = sinkhole" when you define batch inputs.
* This setting causes the input to load the file destructively.

Hope this helps ...

cheers, MuS

deepashri_123
Motivator

Hey@MuS,

What permission does the file require for destructive input?

0 Karma

MuS
SplunkTrust
SplunkTrust

If on nix you need write (+w) permissions, and parent directory should be accessible (+x) to the user which is you want to have delete permission.

If on Windows ... ¯\_(ツ)_/¯ sorry cannot help here, but I'm sure you will find something asking google.

cheers, MuS

jiaqya
Builder

Mus, Thanks, thats mostly what i want. But , is there an option to delete only beyond 7 days or 'n' number of days , so at least i retain few recent files...

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...