Splunk Search

Need help crafting an efficent solution for deleting 2000 events...

lyndac
Contributor

Using Splunk 6.3.1. I have been given a list of about 2000 events that need to be "deleted" from my index. (I do understand that using |delete only hides the data...that is fine). I will be given a similar list at least weekly and possibly daily. Obviously I need to automate this, so I'm thinking a python script using the rest interface to execute a search. My question: what is the most efficient way to craft the search. If I were doing them one at a time, I would do

search index=foo identifier=theidentifierofmyevent | delete

But, I'm concerned that issuing 2,000 such commands would really tie up the search head. But, putting 2000 OR'd search terms possibly isn't the best idea either.

The file I receive is a text file with two space-separated columns, the first column is the field name and the second is the value to search for.

Any suggestions as to the best way to handle this? I guess as long as I wait for one search to finish, before starting another, it should be ok to do the searches one at a time. I just want to be as efficient as possible.

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

I would create a lookup table file from the text file OR ingest the text file, which contains identifiers, and then use a subsearch to extract those filters and delete. This you would be done in one search per day/week.

sample search
With lookup eventsToBeDeleted.csv contains one column identifier with 2000 rows

index=foo [| inputlookup eventsToBeDeleted.csv | table identifier] | delete

With text file ingested in SPlunk and field extractions are configure

index=foo [search index=something sourcetype=something | table identifier ] | delete

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

I would create a lookup table file from the text file OR ingest the text file, which contains identifiers, and then use a subsearch to extract those filters and delete. This you would be done in one search per day/week.

sample search
With lookup eventsToBeDeleted.csv contains one column identifier with 2000 rows

index=foo [| inputlookup eventsToBeDeleted.csv | table identifier] | delete

With text file ingested in SPlunk and field extractions are configure

index=foo [search index=something sourcetype=something | table identifier ] | delete
0 Karma

jkat54
SplunkTrust
SplunkTrust

Honestly I would go with the OR approach so that it's one search.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...