1) Your other splunk instances will not automatically hook up to the deployment server, you will need to point them to it. (http://docs.splunk.com/Documentation/Splunk/latest/Deploy/ConfigureDeploymentClients)
What this means is, regardless of the changes made on the deployment server, nothing takes affect on any other splunk instances.
Secondly, your deployment configuration should be configured in stages. You specify what hosts you would like to apply the configuration to in the serverclass.conf. (http://docs.splunk.com/Documentation/Splunk/latest/Admin/Serverclassconf)
Take note of the machine whitelist and blacklists. This is a useful way to configure indexers to receive some apps, search heads others. So on and so forth. There are multiple ways to do this, it is worth spending the time to figure out what works for your deployment.
2) unless you specify a change in the deployment client, nothing is going to change by simply setting up a deployment server. Also note splunk's order of precedence. (http://docs.splunk.com/Documentation/Splunk/latest/admin/Wheretofindtheconfigurationfiles)
(Least precedence) System default < app default < app local < System local (Greatest precedence) (http://docs.splunk.com/Documentation/Splunk/latest/admin/Wheretofindtheconfigurationfiles#Example_of_how_attribute_precedence_works)
3) its always a great idea to test things before implementing them. Test a forwarder, search head and indexer if feasible.
4) This would be accomplished by hooking them up one by one and applying proper whitelisting and blacklisting.
For example - Have a forwarder app that manages your forwarder outputs. You can set up multiple apps if you require different outputs.conf settings depending on the location in your network. Manage what forwarders can connect with whitelists and blacklists.
Same applies to inputs - Anything that is universal for all hosts of one type should be in one app. Anything that is specific to your one forwarder should then be moved to another. Again, manage with whitelists and blacklists.
The idea with your deployment server is to eventually get to a point where adding new systems, inputs and servers into splunk is as easy as pointing the new install to the deployment server. With this in mind, a little planning is required to make sure your serverclass.conf file is easy to read and logical. Your naming scheme for splunk instances becomes very important too!
I hope this helps.
This is by far no means a comprehensive guide to the deployment server.
... View more