Splunk Search

How do I clean metadata after deleting the eventdata for a particular source

sanurd
Path Finder

Hello,

I indexed data using files and directory monitor to index multiple files in a folder.
I later deleted the data from the source using this search.

source="C:\splunkproject\perf\36257\*" host="AUSD1C" sourcetype="perf" | delete

I expected this to not only delete the events but also the metadata. Looks like the metadata for this source is still present.

When I run the following search, I see the event count is 0 but the metadata for the source is still present.

| metadata type=sources | where (source LIKE "%36257%")

firstTime lastTime recentTime source totalCount type
2147483647 0 1433208419 C:\splunkproject\perf\36257\A_perf_mon.log 0 sources
2147483647 0 1433208419 C:\splunkproject\perf\36257\B_perf_mon.log 0 sources

Any ideas on how to clean up the meta data.

I looked at 'splunk help clean' which doesn't have an option to clean metadata and the following post,

http://answers.splunk.com/answers/227046/how-reliable-is-the-metadata-command-and-how-can-i.html?utm...

which seems to indicate that the metadata won't be cleaned up using | delete search command.
Is there a way besides deleting the whole index ( which I don't want to do as I have other sources that I want to use )

thank you in advance.

Tags (2)
1 Solution

MuS
SplunkTrust
SplunkTrust

Hi sanurd,

Looks like the metadata for this source is still present. Yes it is, because the delete command does NOT delete events. The events are no longer searchable, but are still in the index - see docs http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/Delete

If you want to remove events and its metadata you have to use clean - but be aware this can only be done on index level.
Another useful docs link includes the hint btw http://docs.splunk.com/Documentation/Splunk/6.2.3/Indexer/RemovedatafromSplunk#Delete_events_from_su...

The delete operator does not update the metadata of the events, so any metadata searches will still include the events although they are not searchable. The main All indexed data dashboard will still show event counts for the deleted sources, hosts, or sourcetypes.

Hope that helps ...

cheers, MuS

View solution in original post

MuS
SplunkTrust
SplunkTrust

Hi sanurd,

Looks like the metadata for this source is still present. Yes it is, because the delete command does NOT delete events. The events are no longer searchable, but are still in the index - see docs http://docs.splunk.com/Documentation/Splunk/6.2.3/SearchReference/Delete

If you want to remove events and its metadata you have to use clean - but be aware this can only be done on index level.
Another useful docs link includes the hint btw http://docs.splunk.com/Documentation/Splunk/6.2.3/Indexer/RemovedatafromSplunk#Delete_events_from_su...

The delete operator does not update the metadata of the events, so any metadata searches will still include the events although they are not searchable. The main All indexed data dashboard will still show event counts for the deleted sources, hosts, or sourcetypes.

Hope that helps ...

cheers, MuS

jaredlaney
Contributor

The delete command has never worked predictably and always causes the metadata and actual data to get out of sync. I wish Splunk would just deprecate the command because it doesn't work.

sanurd
Path Finder

Thanks MuS for your response. I was hoping to find a way to delete events and metadata for a particular source without deleting the whole index. Looks like it is not possible , Thanks anyways.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...