We have 2 peers that each forwarder load balances between so there is roughly 50% of the primary data on each one. The load on the peers is thus evenly spread. The replciation factor of 2 so each peer works as a back up for the other.
Doing a search I can see that split on the "splunk server" field is roughly 50 - 50 which is exactly what I want to see.
During routine maintenance one of the peers is brought offline (splunk offline command) . While it is down the primaries are reassigned to the other peer as you would expect. Looking at "splunk server" from a search I can see 100% of the data on the remaining peer so the resilancy works. The problem is that when the peer is brought back online there is still 100% of the primary data on peer that was left up. This means that one peer is getting hammered while the other does nothing. Is there any way to force splunk to reassign 50% of the primaries back to the peer that was down? I am aware that the data is on the peer but it is now a replicate and does not participate in the searches.
