Getting Data In

Can I forward to 2 splunkcloud deployments at a time ?

yannK
Splunk Employee
Splunk Employee

I have 1 splunkcloud deployment, and I need to send a copy of my data to another deployment.

  • can the splunklcoud indexers forward data to another deployment ?
  • can my forwarder send data to both deployments ?

I tried to see the forwarder credential apps I have, but they do not play nice together.

Tags (3)
1 Solution

yannK
Splunk Employee
Splunk Employee

After testing here is the procedure to configure a forwarder to send to 2 groups of indexers.

1- download/retrieve the splunkcloud forwarder credential apps for the deployment A and B
rename the app folder to distinguish them.
by example: splunkcloudforwarder_A splunkcloudforwarder_B
You need to keep them both, as they contains distinct ssl certificates.

2- in the app, go to default/outputs.conf
and edit the name of the tcpout group to distinguish them

[tcpout:primary_indexers]
to
[tcpout:primary_indexers_A]
and
[tcpout:primary_indexers_B]

3 - add a local/outputs.conf in one of the apps ( or in etc/system/local, but it's not convenient to deploy in apps)
and put the 2 new groups as default destination groups to clone the data to both.
[tcpout]
defaultGroup = primary_indexers_A, primary_indexers_B

4 - test the merging of the configurations with a btool command
cd $SPLUNK_HOME/bin
./splunk cmd btool outputs list
You want to see
[tcpout]
defaultGroup = primary_indexers_A, primary_indexers_B
and 2 groups
[tcpout:primary_indexers_A]
[tcpout:primary_indexers_B]

if you do not see them, use
./splunk cmd btool outputs list --debug
to check where each configurations are coming from.

5- start the forwarder and confirm that it is sending data to the 2 groups
you can look at the internal logs (index=_internal host=myforwardername)

Remarks :

  • when you start splunk, the clear ssl password in the apps /default/outputs.conf will be encrypted and saved in /local/outputs.conf file. But cannot be decrypted by another forwarder. So if you want to copy the apps from a forwarder to another, (or deploy it using a deployment server), make sure to remove the line with the local folder.

  • You can use this configuration on the forwarders directly

  • If you want to use intermediary forwarder you just need to configure your first forwarders to send the data to the intermediary forwarders, and setup an input on the intermediary forwarder in inouts.conf
    [splunktcp:9997]

    • You can use an Universal forwarder or a lightweight forwarder as intermediary forwarder (the heavy forwarder allow parsing and filtering but has a heavier load, and require you to install all your indexers parsing apps on them too)

View solution in original post

yannK
Splunk Employee
Splunk Employee

After testing here is the procedure to configure a forwarder to send to 2 groups of indexers.

1- download/retrieve the splunkcloud forwarder credential apps for the deployment A and B
rename the app folder to distinguish them.
by example: splunkcloudforwarder_A splunkcloudforwarder_B
You need to keep them both, as they contains distinct ssl certificates.

2- in the app, go to default/outputs.conf
and edit the name of the tcpout group to distinguish them

[tcpout:primary_indexers]
to
[tcpout:primary_indexers_A]
and
[tcpout:primary_indexers_B]

3 - add a local/outputs.conf in one of the apps ( or in etc/system/local, but it's not convenient to deploy in apps)
and put the 2 new groups as default destination groups to clone the data to both.
[tcpout]
defaultGroup = primary_indexers_A, primary_indexers_B

4 - test the merging of the configurations with a btool command
cd $SPLUNK_HOME/bin
./splunk cmd btool outputs list
You want to see
[tcpout]
defaultGroup = primary_indexers_A, primary_indexers_B
and 2 groups
[tcpout:primary_indexers_A]
[tcpout:primary_indexers_B]

if you do not see them, use
./splunk cmd btool outputs list --debug
to check where each configurations are coming from.

5- start the forwarder and confirm that it is sending data to the 2 groups
you can look at the internal logs (index=_internal host=myforwardername)

Remarks :

  • when you start splunk, the clear ssl password in the apps /default/outputs.conf will be encrypted and saved in /local/outputs.conf file. But cannot be decrypted by another forwarder. So if you want to copy the apps from a forwarder to another, (or deploy it using a deployment server), make sure to remove the line with the local folder.

  • You can use this configuration on the forwarders directly

  • If you want to use intermediary forwarder you just need to configure your first forwarders to send the data to the intermediary forwarders, and setup an input on the intermediary forwarder in inouts.conf
    [splunktcp:9997]

    • You can use an Universal forwarder or a lightweight forwarder as intermediary forwarder (the heavy forwarder allow parsing and filtering but has a heavier load, and require you to install all your indexers parsing apps on them too)
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...