Getting Data In

Why impliment load balancing on my forwarder if all my FW data is being forwarded to all of the indexers

mhouse3
Path Finder

If I have my outputs.conf file on all of my forwarders are configured to send all the data to all of the indexers what is the point or value of load balancing on the forwarder?

I would think that autoLB would not need to equal true of all the data is going to all of the indexers.

0 Karma

stboch
SplunkTrust
SplunkTrust

This really depends on your environments configuration. If your indexers are separate where you are using indexer clustering but want to have multiple copies of the data

output.conf config example.

[tcpout]
defaultGroup=indexer1,indexer2
[tcpout:indexer1]
server=10.1.1.197:9997
[tcpout:indexer2]
server=10.1.1.200:9997

Then there is no load balancing to take place as you only have data going to each of your indexers at the same time.

However if you have indexer clustering where your indexers are making copies of your data for you. Or you have no need to duplicate logs to multiple environments using this example with load balancing prevents several forwarders from sending data to one indexer because indexers have no communication to the clients that they are loaded other than to stop receiving data.

Outputs.conf Example:

[tcpout]
defaultGroup=my_indexers

[tcpout:my_indexers]
server=mysplunk_indexer1:9997, mysplunk_indexer2:9996
0 Karma

somesoni2
Revered Legend

Are you doing data load balancing (data is distributed across multiple indexers where each indexer gets a portion of the data) OR data cloning (same data is send to multiple indexers where each indexer gets full data, thus duplicating the data)? You can see the outputs.conf configuration difference in these two cases here:

http://docs.splunk.com/Documentation/Forwarder/7.0.2/Forwarder/Configureforwardingwithoutputs.conf#C...

OR you can share your outputs.conf configuration (mask any servernames) and point us with which attribute/line you think is not required.

0 Karma

mhouse3
Path Finder

Data load balancing at the forwarder.

So given that I have my outputs.conf file on all of my forwarders configured to send all the data to all of the indexers what is the point or value of "DATA" load balancing on the forwarder?

0 Karma

somesoni2
Revered Legend

The forwarder does the data load balancing automatically. Provided your outputs.conf setting has all indexers as described in the link above part of single LB group (stanza starting with tcpout:), it's doing the load balancing. It's not sending all the data to all the indexers, it's spreading the data to multiple indexers so that you've higher availability of data (no single point of failure) and putting less load on a single indexer. If a LB group (defined in outputs.conf) has only a single indexer, then you're not doing Data load balance, that's cloning.

0 Karma
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us in this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...