I have a summary index that's distributed among three indexers. It is populated by searches that run from a single search head. I wanted to clean its eventdata, so I stopped splunkd on the search head and ran splunkd clean eventdata -index summary_perfmon. The command line said it was cleaning the index and threw no error, but when I restarted Splunk I found all the events were still there. What is the proper procedure for cleaning a distributed index? Do I need to stop Splunk on the search head and all related indexers and run the clean command on each one?
You'd need to run this on any place where the summary index exists and contains data. If the search head is the only place where the summary index exists, running the command on that instance should be sufficient. It sounds as if you've got data on the indexers as well, so you'll need to run it on those instances. There is no difference between cleaning data from a summary index or another type of index. In fact, there really isn't any difference between a summary index and a non summary index at all aside from the data that you're storing.
You'd need to run this on any place where the summary index exists and contains data. If the search head is the only place where the summary index exists, running the command on that instance should be sufficient. It sounds as if you've got data on the indexers as well, so you'll need to run it on those instances. There is no difference between cleaning data from a summary index or another type of index. In fact, there really isn't any difference between a summary index and a non summary index at all aside from the data that you're storing.
You can bring them down one at a time, that's fine. You don't need them all down to clean the index as they're really separate indexes with the same names.
@jbsplunk, thanks for the reply. In general, to clean a distributed index, do I have to bring down Splunk on all indexers at once? Or could I bring down Splunk on each indexer one at a time, so that any data coming in could go to the remaining indexers?