As the Cluster Deployments are reaching maturity, we are planning to add a new Cluster Peer/Indexer to the existing Cluster Master, especially when currently available clustered indexers reach Disk Storage Capacity. When the New Indexers are added, forwarders are configured to stop sending the data to old indexers, and data is INDEXED on the newly added Cluster Peer. Unfortunately, data is still getting replicated to the Legacy Cluster Peer which results in unintended consequences and causes the disk space to fill up on clustered indexers.
Ideally, we would prefer if the Cluster Master could calculate the available disk on each indexer and make informed decisions during replication, and not replicate data if some peer is close to Disk Storage Capacity. Otherwise, Splunk should at least provide a Capability to manually exclude some peers from replication, either from the UI or using a REST call.
