Is it possible to change default indexer IP on universal forwarders using Deployment Server?

[att35]

Hi,

We have multiple indexers within our Splunk infrastructure each receiving logs from Splunk Forwarders within their respective data centers. Each forwarder is also connected to a single Deployment Server.

One of our indexer is catering to a lot more forwarders along with several other inputs and this has started to cause a resource issues. To alleviate the load, we added another indexer in the same network and would like to move some forwarder over to this new instance.

Is something like this possible via Deployment Server? Can we change the default outputs.conf setting(or find a way to override it)?

Thanks,

~ Abhi

[woodcock]

You will have to build a script to delete (or move) the $SPLUNK_HOME/etc/system/local/outputs.conf file and deploy a valid $SPLUNK_HOME/etc/system/apps/SomeAppNameHere/default/deploymentclient.conf file and then restart splunk. Make sure that before you deploy this app, that you have your Deployment Server configured to push out the updated outputs.conf app. This app will give you a framework for everything that you will need to do, including the automatic running of a script to execute arbitrary shell commands.

https://splunkbase.splunk.com/app/2722/

[thambisetty]

Hi @abhijittikekar,

you can do this by deployment server.

you just create an app and create outputs.conf in default/local directory. example outputs.conf below:

[tcpout]
defaultGroup = default-autolb-group
[tcpout:default-autolb-group]
server= indexer_ip:9997

replace indexer_ip with your indexer IP.

In Deployment server - settings -forwarder management and change app settings in app tab by choosing newly created app enable restart.create a server class in forwarder management make this new app and the forwarder where you would like to deploy this app.

finally remove old app which has outputs.conf

[att35]

Hi thambisetty,

Thanks for this input.

Problem we have is that till now, these forwarders were not managed by Deployment Server. The installation/configuration was all done locally on each server which means that the current indexer setting for each of these resides in \etc\system\local\outputs.conf. These forwarders do not have any app deployed yet and therefore we do not have any old app to remove.

If we do push a new app as you mentioned above, which [tcpout] stanza takes precedence? the one in the new app managed by Deployment Server or the one under \etc\system\local\outputs.conf?

Thanks,

~ Abhi

[thambisetty]

System/local will have highest precedence Based on the documentation. Even if you deploy outputs.conf through deployment server you will not see changes in outputs.conf on the deployment client.
I'll simulate this and try to give you an update on this.

[thambisetty]

As per the answer given in the below link, says that system/local/outputs.conf will be overridden with outputs.conf deployed from deployment server.
https://answers.splunk.com/answers/65791/changing-uf-outputs-conf-using-deployment-server.html

Try to deploy app as I said in my answer.

[JDukeSplunk]

Yes.

We have a few apps depending on if the SF needs to output to a heavy forwarder, or directly to the indexer.

So, make a new app with a /local/outputs.conf and deploy it to the clients you wish.

[tcpout:GROUPNAME]
server = HOSTINDEXER1.DOMAIN.COM:9997,HOSTINDEXER2.DOMAIN.COM:9997

[tcpout]
defaultGroup = GROUPNAME

You may also need an app.conf file in /local. Not sure...

# Autogenerated file 
[install]
state = enabled

[wildbird]

Hey

We have reached a point where we need to expand the number of servers in our architecture For reasons of proper management, This will require me to move HF from the existing address to another address.

currently the HF is part of SHC, I would love an explanation of how the transition can be done smoothly