Getting Data In

Indexing the same file twice

abhayneilam
Contributor

Hi,

I have an index called "XYZ" and in it i have a file called "abc.txt" and I am taking the help of a configuration file to import "abc.txt" in splunk automatically. Now , I have by mistaken copied it twice in the same location where my splunk pics up the data "abc.txt" for indexing. Now, I am getting double of data, plz help me to delete the data which had been indexed twice. My Index is still showing the duplicate data even after removing the duplicate file from the location.

Regards,
Abhay

Tags (2)
0 Karma
1 Solution

lukejadamec
Super Champion

Once the data is index it will stay there unless you delete it, and even then it is still there, but you can't see it.

You can remove the duplicate data with the delete command.

Create a search that give you results that exactly match the data you want to delete, and then add delete at the end of the search and re-run it. This time the search will execute as before, but instead of showing you results it will delete the data from the index.

You can read about the delete command here:
http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/Delete

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

Since you copied your abc.txt file to same location, it must have gotten renamed (to abc_copy.txt or something similar). Execute following command to delete the duplicate data.

index=XYZ source=abc_copy.txt | delete

The user which will execute this search should have 'can_delete' capabilities added.

0 Karma

abhayneilam
Contributor

Many Many thanks for demonstrating me the whole scenario.

0 Karma

lukejadamec
Super Champion

Once the data is index it will stay there unless you delete it, and even then it is still there, but you can't see it.

You can remove the duplicate data with the delete command.

Create a search that give you results that exactly match the data you want to delete, and then add delete at the end of the search and re-run it. This time the search will execute as before, but instead of showing you results it will delete the data from the index.

You can read about the delete command here:
http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/Delete

abhayneilam
Contributor

Many thanks for the reply !!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...