When I try and restart one of my indexers after an OS upgrade I am seeing the following messages. My 2 other indexers are up and running. How do I fix this. I found one articale where they talk about fixing the offending buckets but don't say how and I am not positive this is the same issue
09-06-2018 06:37:17.576 -0400 ERROR DatabaseDirectoryManager - idx=main bid=main~392~F18EA0F4-48F1-4D8C-8209-5B40F
0B66E1E bucket=392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E Detected directory manually copied into its database, caus
ing id conflicts [path1='/opt/splunk/var/lib/splunk/defaultdb/db/rb_1535649215_1535592644_392_F18EA0F4-48F1-4D8C-8
209-5B40F0B66E1E' path2='/opt/splunk/var/lib/splunk/defaultdb/db/392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E'].
09-06-2018 06:37:17.579 -0400 ERROR IndexerService - Error intializing IndexerService: idx=main bid=main~392~F18EA
0F4-48F1-4D8C-8209-5B40F0B66E1E bucket=392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E Detected directory manually copied
into its database, causing id conflicts [path1='/opt/splunk/var/lib/splunk/defaultdb/db/rb_1535649215_1535592644_
392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E' path2='/opt/splunk/var/lib/splunk/defaultdb/db/392_F18EA0F4-48F1-4D8C-82
09-5B40F0B66E1E'].
09-06-2018 06:37:17.584 -0400 FATAL IndexerService - One or more indexes could not be initialized. Cannot disable
indexes on a clustering slave.
Hi @a238574,
Based on logs there is Bucket ID conflict in your main
index.
You have Bucket ID 392
at /opt/splunk/var/lib/splunk/defaultdb/db/rb_1535649215_1535592644_392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E
and /opt/splunk/var/lib/splunk/defaultdb/db/392_F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E
in same index which is causing issue here. The thing is bucket starting with rb_
is a replicated bucket and should replicated from different indexer but here it looks like you have same GUID - F18EA0F4-48F1-4D8C-8209-5B40F0B66E1E
across different indexers. Each Indexer should have different GUID
so first check across your indexer with duplicated GUID.