Deployment Architecture

After removing a temporary search head without removing it first as a search head cluster member, how to troubleshoot error "CONFIGURATION ID MISMATCH"?

Raghav2384
Motivator

Hey Experts,

I have a cluster setup with 3 search heads and 1 deployer. Out of the three search heads, one was temporary. Once i got the permanent, i being stupid and without thinking, removed the temporary without removing it being part of the cluster members. I decided to redo the whole deployer setup. Got deployer set, initialized cluster members done. Can't elect captain as i get this message:

In handler 'shclustermemberconsensus': CONFIGURATION ID MISMATCH

When i looked in the splunkd.log, it was looking for the temporary cluster member which was a part of it initially. Now it's gone. I just can't elect a captain. Is there any other way I can convince the SH cluster member I want to be captain that the temporary SH I added earlier is no more and R.I.P?

Thanks,
Raghav

1 Solution

Raghav2384
Motivator

I had to do a ./splunk stop on all instances, clean and re-configure. Glad those were brand new sets. Just an FYI, if you want you remove/replace an existing cluster member, please follow the documentation.

http://docs.splunk.com/Documentation/Splunk/6.2.1/DistSearch/Removeaclustermember

splunk remove shcluster-member -mgmt_uri <URI>:<management_port>

Thanks,
Raghav

View solution in original post

Raghav2384
Motivator

I had to do a ./splunk stop on all instances, clean and re-configure. Glad those were brand new sets. Just an FYI, if you want you remove/replace an existing cluster member, please follow the documentation.

http://docs.splunk.com/Documentation/Splunk/6.2.1/DistSearch/Removeaclustermember

splunk remove shcluster-member -mgmt_uri <URI>:<management_port>

Thanks,
Raghav

Lucas_K
Motivator

Minimum members for a shc is 3. A captain normally won't be elected with only 2 members as there isn't a deciding vote.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...