We have three Node Search Head Cluster member and we are seeing the following messages in our splunkd.log:
12-08-2015 10:00:25.468 -0800 WARN ConfMetrics - single_action=PUSH_TO took wallclock_ms=1553! Consider a lower value of conf_replication_max_push_count in server.conf on all members
12-08-2015 10:00:25.468 -0800 WARN ConfMetrics - single_action=PUSH_TO took wallclock_ms=1553! Consider a lower value of conf_replication_max_push_count in server.conf on all members
What does this error message mean?
Here is some information on this error message
in summary, Configuration replication within the SHC is slower than the ideal standards (of 1 second), possibly because the SHC needs to frequently replicate the config updates with large lookup files.
We currently don't support blacklisting of the lookup files in the SHC.If you want to confirm this, on each of the Search Heads, please take a backup of the latest bundle file under var/run/splunk/snapshot to a temporary directory, and make a note of their sizes. untar the bundle file (tar xvf ...) and make a note of the largest files in the bundle directory. The expectation is the largest files in the bundle would be the lookup csv files, which will confirm this.
Here is some information on this error message
in summary, Configuration replication within the SHC is slower than the ideal standards (of 1 second), possibly because the SHC needs to frequently replicate the config updates with large lookup files.
We currently don't support blacklisting of the lookup files in the SHC.If you want to confirm this, on each of the Search Heads, please take a backup of the latest bundle file under var/run/splunk/snapshot to a temporary directory, and make a note of their sizes. untar the bundle file (tar xvf ...) and make a note of the largest files in the bundle directory. The expectation is the largest files in the bundle would be the lookup csv files, which will confirm this.