Getting Data In

Indexing the same file twice

abhayneilam
Contributor

Hi,

I have an index called "XYZ" and in it i have a file called "abc.txt" and I am taking the help of a configuration file to import "abc.txt" in splunk automatically. Now , I have by mistaken copied it twice in the same location where my splunk pics up the data "abc.txt" for indexing. Now, I am getting double of data, plz help me to delete the data which had been indexed twice. My Index is still showing the duplicate data even after removing the duplicate file from the location.

Regards,
Abhay

Tags (2)
0 Karma
1 Solution

lukejadamec
Super Champion

Once the data is index it will stay there unless you delete it, and even then it is still there, but you can't see it.

You can remove the duplicate data with the delete command.

Create a search that give you results that exactly match the data you want to delete, and then add delete at the end of the search and re-run it. This time the search will execute as before, but instead of showing you results it will delete the data from the index.

You can read about the delete command here:
http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/Delete

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

Since you copied your abc.txt file to same location, it must have gotten renamed (to abc_copy.txt or something similar). Execute following command to delete the duplicate data.

index=XYZ source=abc_copy.txt | delete

The user which will execute this search should have 'can_delete' capabilities added.

0 Karma

abhayneilam
Contributor

Many Many thanks for demonstrating me the whole scenario.

0 Karma

lukejadamec
Super Champion

Once the data is index it will stay there unless you delete it, and even then it is still there, but you can't see it.

You can remove the duplicate data with the delete command.

Create a search that give you results that exactly match the data you want to delete, and then add delete at the end of the search and re-run it. This time the search will execute as before, but instead of showing you results it will delete the data from the index.

You can read about the delete command here:
http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/Delete

abhayneilam
Contributor

Many thanks for the reply !!

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...