Deletion of event data in a index for performance

[linu1988]

Hello,
I would like to know if deletion of events which are not required will increase the search performance? They are in very big numbers which slowed my search down on the dashboard.

If not do i have to clean the existing index or do we have some other solution?

Thanks

[mloven_splunk]

Deletion of data (via the 'delete' command) won't increase performance. It's sort of a misnomer. The 'delete' command won't actually delete any data from your indexes, it will only make the data 'invisible' to searches.

Cleaning out an index is certainly an option, but a drastic one. If you don't mind losing ALL data from your index, you can go that route.

I'd start looking at the underlying causes of WHY your searches are slow.

Are you piping everything into one index? Maybe look at separating your data into different indexes. This should make searches (prepended with index=) run a bit faster.

Over what time range are you running your searches? If you're constantly running searches "over all time", then you should get out of that habit. Only run a search over the time range you need.

How many scheduled saved searches do you have running? If you're running Splunk on an underpowered server, your ad-hoc search may be contending with scheduled saved searches (or other users running ad-hoc searches) for CPU cycles.

[aelliott]

alternatively you could set "expiration" times, or expiration per amount of data, by default data is stored for 6 years.
http://answers.splunk.com/answers/4236/how-to-deleteoverwrite-data-older-than-x-number-of-days

http://docs.splunk.com/Documentation/Splunk/6.0/Indexer/Setaretirementandarchivingpolicy

[linu1988]

The platform/the dashboard configuration isn't a problem. I wouldn't be so happy to reset the index by which I would loose my required data.

I can't separate the index as all were the same set of data of similar log. However due to some test logs million of records are now present in the index, which is the cause of performance that I understand. When showing the specific source of data of a given category now it's taking very long hence I was thinking of deleting the records. Thank you for your suggestion!!!