Can the Cluster Peer be re-added to the Cluster Master without restarting Cluster master or the Cluster Peer?
I have three Cluster Peers with RF=2 and SF=2. For some reason, one warm bucket of one of the indexes is not getting replicated to other peers. The data in the bucket is searchable.
Answer to your question is yes. In some rare condition it may be required that Cluster Peer may be re-added to the Cluster Peer. Usually this step should be done with Splunk Support Consult.
On re-add the existing cluster peer to cluster master you can issue curl command like below
curl -k --basic -u admin:changeme -d clearMasks=false https://<peerIP>:<mgmtPort>/services/cluster/slave/control/control/re-add-peer
Where peerIP:mgmtPort refer to cluster peer that needs to be re-added.
ii) Or, other choice will be to run the following command on the Cluster Peer that needs to be re-added.
splunk _internal call /services/cluster/slave/control/control/re-add-peer –method
Only situation I have found it to be useful is when due to some reason bucket state gets mismatched between Cluster Master and Cluster Peer. When Cluster Peer will join to the CM, it will report all buckets information to the Cluster master and that will clear mismatch status information on cluster master.