- Splunk Answers
- :
- Splunk Administration
- :
- Deployment Architecture
- :
- How to sync the cluster members of a search head?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How to sync the cluster members of a search head?
I have message popping on my UI messages " Search head cluster member (sh3) is having problems pushing configurations to the search head cluster captain (sh4). Changes on this member are not replicating to other members. We have 4 search heads with dynamic captain
I have ran the ./splunk resync shcluster-replicated-config command and also ran the /splunk clean kvstore --local . Still it is showing the error
Any other workaround for this
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can you try reducing the value of conf_replication_max_push_count and give it some time to catch up with the syncing in small pushes only.
At below post, its been resolved this way -
https://answers.splunk.com/answers/539335/why-is-a-search-head-cluster-member-not-replicatin.html
Please accept as answer if it resolves your case.
Thanks
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Where can I change the conf_replication_max_push_count ,should I do It on the captain and push the changes?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
you can set it at server.conf. A direct manual update to the .conf file would not be replicated but however through Web OR CLI OR REST they should be.
Check this for SHC replication -
https://docs.splunk.com/Documentation/Splunk/7.3.0/DistSearch/HowconfrepoworksinSHC
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I see the following message in the logs
ERROR ConfReplicationThread - Error pushing configurations to captain=https://hesplsrhc003com:8089, consecutiveErrors=173 msg="Error in acceptPush: Non-200 status_code=400: ConfReplicationException: Cannot accept push with outdated_baseline_op_id=d211555c171b47ccc8c7fe5f5588856096680c0e; current_baseline_op_id=c6235cd928e2196f63bb3cb292a55a2687467c46": Search head clustering: Search head cluster member (https://hesplsrhc001.com:8089) is having problems pushing configurations to the search head cluster captain (https://hesplsrhc003.com:8089). Changes on this member are not replicating to other members.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@vrmandadi Check the splunkd logs for the resync if it happened correctly or there was some error.
Also this post although is for captain but may help - https://answers.splunk.com/answers/525189/how-to-resolve-error-error-pulling-configurations.html
Introducing Splunk Enterprise 9.2
Adoption of RUM and APM at Splunk
Routing logs with Splunk OTel Collector for Kubernetes
