Splunk Search

Remove data from Index

efelder0
Communicator

I have indexed many months worth of data, but would like to "remove" only the first of the 3 months worth of data. However, I cannot clean out the entire index. Is this possible with the clean eventdata command?

Tags (1)
0 Karma
1 Solution

MuS
SplunkTrust
SplunkTrust

Hi efelder0

you can search for the data you no longer need and append

| delete 

to it. This data will then no longer searchable but still is in the index.

If this helps.

Cheers,
MuS

View solution in original post

MuS
SplunkTrust
SplunkTrust

Hi efelder0

you can search for the data you no longer need and append

| delete 

to it. This data will then no longer searchable but still is in the index.

If this helps.

Cheers,
MuS

jtashiro
New Member

Can someone as Splunk explain the purpose of "delete" command, if it doesn't actually delete data from an index, but makes it un-searchable. As I understand it, "delete" operation is irreversible, the deleted data continues to consume disk space, and there is no way to free that up? Doesn't make sense to me. Am I not understanding it?

0 Karma

botkindl
Explorer

It's very useful in some cases. For instance, we had an issue where logrotate was rotating syslogs and Splunk was indexing them (fixed with a blacklist entry). Users were getting totally confused by the "extra" hostnames, which were actually filenames from the rotated files -- and the log messaegs were duplicated as well. So we ran a search, piped to delete, everyone is happy.

At the same time, I don't ever have to explain (or defend) to our internal audit folks how and why we can actually delete data. No matter if we delete it or not, it's still there in the rawdata files and still can be found if needed. I think it's a good compromise of being able to remove extraneous/distracting search results, and being able to say that the data is permanent.

aaraneta_splunk
Splunk Employee
Splunk Employee

Hi @jtashiro,

Have you checked out the accepted answer at this link? It may be a good place to start.

However, if you are not satisfied with that explanation, I would suggest posting a new question about this topic since this post is from over 3 years ago and may not get the visibility you would like in order to help you.

0 Karma

jtashiro
New Member

I've read the accepted answer, and it doesn't satisfy my question. The question is best answered by Splunk technical team, with insight into why 'delete' was built to hide/mask data, but not actually 'delete' it and free up space. The 'delete' command is inaccurately and poorly named.

0 Karma

linu1988
Champion

Adding to that, metadata will be still be available. That can't be removed with delete..

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...