How can I migrate data from stand-alone indexers into a cluster under 6.6.2?

[daniel333]

All,

I have 10 servers that are end of life. So I am replacing them with 10 new ones. I stood the new ones up as an index cluster and things went well. I sorta assumed I could copy the data from each indexer, reboot and the cluster woudl pick it up. This does not seem to be the case as I am getting the message below.

08-15-2017 18:11:34.645 +0000 WARN  CMSlave - Failed to register with cluster master reason: failed method=POST path=/services/cluster/master/peers/?output_mode=json master=mymasterserver.mydomain.com:8089 rv=0 gotConnectionError=0 gotUnexpectedStatusCode=1 actual_response_code=500 expected_response_code=2xx status_line="Internal Server Error" socket_error="No error" remote_error=Cannot add peer=X.x.x.x mgmtport=8089 (reason: bucket already added as clustered, peer attempted to add again as standalone. guid=FAF83722-06AA-4085-BB7A-750447C6FEF9 bid= _telemetry~1~FAF83722-06AA-4085-BB7A-750447C6FEF9). Make sure pass4SymmKey is matching if the peer is running well. [ event=addPeer status=retrying AddPeerRequest: { _id= active_bundle_id=0EC2D631C0BA7AB84B04E88C8E7B437B add_type=Initial-Add base_generation_id=0 batch_serialno=1 batch_size=1 forwarderdata_rcv_port=9997 forwarderdata_use_ssl=0 last_complete_generation_id=0 latest_bundle_id=0EC2D631C0BA7AB84B04E88C8E7B437B mgmt_port=8089 name=FAF83722-06AA-4085-BB7A-750447C6FEF9 register_forwarder_address= register_replication_address= register_search_address= replication_port=8080 replication_use_ssl=0 replications= server_name=myserver.mydomain.com site=default splunk_version=6.6.0 splunkd_build_number=4b804538c686 status=Up } ].  

Can someone point me to the correct procedure here?

thanks,
-Daniel

[skoelpin]

It looks like your indexers cant communicate with the cluster master.. You need to update your server.conf on the search peers to point to your master

If you did this already, then it could also be a bad pass4symm key value

[replication_port://9887]

[clustering]
master_uri = https://10.152.31.202:8089
mode = slave
pass4SymmKey = whatever

https://docs.splunk.com/Documentation/Splunk/6.6.2/Indexer/Configurepeerswithserverconf

[somesoni2]

My suggestion would be to start reading from this:
http://docs.splunk.com/Documentation/Splunk/6.6.2/Indexer/Migratenon-clusteredindexerstoaclustereden...