We have a search that captures the count per host every 10 minutes and puts the results into a summary index. For some reason an event for a single host count during a 10 minute time period was indexed 27 times. Every event is exactly the same. So in order to correct our stats, 26 of these events need to be deleted...How do i do that? I tried:
... | head 26 | delete
This returned an error: Error in 'delete' command: This command cannot be invoked after the non-streaming command 'head'.
Again, I have 27 events that are exactly the same. How can i delete all except 1? Thanks in advance for any help.
I-Man
I-Man,
If these 27 events are identical and you need to delete 26 of them I would recommend this approach:
1. Determine the "event_id" of the event you would like to keep. To do this, perform your search with the following eval statement to create a unique identifier:
<my 27 events> | eval event_id=splunk_server."@@".index."@@"._cd
2. Delete the other 26 events "NOT event_id=<your_event_id>":
<my 27 events> | eval event_id=splunk_server."@@".index."@@"._cd | search NOT event_id=macfish@@_internal@@2:1309157 | delete
CAUTION: Make sure the first part of your search identifies only the 27 events you are interested in deleting....or the "search NOT event_id=" will delete more than expected.
If you want the last why not just do | tail 1 ?
And with that you have the last event
Error in 'delete' command: This command cannot be invoked after the non-streaming command 'tail'.
To remove duplicate events with different timestamps it is possible using a subsearch and dedup:
sourcetype=fs_notification NOT [search sourcetype=fs_notification | dedup path action modtime sortby +_time] | delete
This will remove duplicated fs change events that have different timestamps, the same path, action and modtime whilst keeping the oldest.
I-Man,
If these 27 events are identical and you need to delete 26 of them I would recommend this approach:
1. Determine the "event_id" of the event you would like to keep. To do this, perform your search with the following eval statement to create a unique identifier:
<my 27 events> | eval event_id=splunk_server."@@".index."@@"._cd
2. Delete the other 26 events "NOT event_id=<your_event_id>":
<my 27 events> | eval event_id=splunk_server."@@".index."@@"._cd | search NOT event_id=macfish@@_internal@@2:1309157 | delete
CAUTION: Make sure the first part of your search identifies only the 27 events you are interested in deleting....or the "search NOT event_id=" will delete more than expected.
Would it be possible to run this search across the board and delete the output from:
soucetype="ds*" | stats count values(host) values(source) values(sourcetype) by _raw | where count > 1
Also, could one run a search and output that into a table and then proceed to do deletions?
Thank you, we are stuck this side!
Perfect...YTMND