Getting Data In

Is it possible to configure load balancing on universal forwarders with preferable servers in tcpout group?

rnr
Path Finder

I'd like to configure universal forwarders on boxes in multiple AZ to forward event to a preferable heavy forwarder located in the same AZ. The problem is word "preferable", universal forwarder doesn't have such settings (would be nice to have based on latency to forwarder for example).

Has anybody tried to build such setup using AWS tools, like Route 53 or ELB?
Router53 provides internal DNS, but I'm not sure if it can resolve names based on request source, that is AZ)

"Preferable" forwarder of course is not the end of the world, but would be nice to have.

--Roman

1 Solution

rnr
Path Finder

For those who interested in configuring local forwarders with prefferable destination and transparent failover, it's quite easy to do with haproxy.

For server located in zoneA:

{standard set of haproxy options}

frontend                            fe-splunkfwd
        mode                            tcp
        option                          tcplog
        log                             global
        bind                            127.0.0.1:7997
        default_backend                 be-splunkfwd


# For splunk forwarders
    backend                             be-splunkfwd
        mode                            tcp
        option                          tcplog
        timeout                         server 15s
        timeout                         connect 2s
        server                          fwd-server-name-zoneA  10.10.19.11:7997 maxconn 8192  check inter 1s
        server                          fwd-server-name-zoneB  10.19.20.11:7997 maxconn 8192  check inter 1s backup

Change backup option in the be-splunkfwd backend respectively for server locate in zoneB.

Of course it would work just fine with ELB, but on the other side local haproxy would give much better control of traffic. Nice to have it for a high logs throughput from host.

View solution in original post

0 Karma

rnr
Path Finder

For those who interested in configuring local forwarders with prefferable destination and transparent failover, it's quite easy to do with haproxy.

For server located in zoneA:

{standard set of haproxy options}

frontend                            fe-splunkfwd
        mode                            tcp
        option                          tcplog
        log                             global
        bind                            127.0.0.1:7997
        default_backend                 be-splunkfwd


# For splunk forwarders
    backend                             be-splunkfwd
        mode                            tcp
        option                          tcplog
        timeout                         server 15s
        timeout                         connect 2s
        server                          fwd-server-name-zoneA  10.10.19.11:7997 maxconn 8192  check inter 1s
        server                          fwd-server-name-zoneB  10.19.20.11:7997 maxconn 8192  check inter 1s backup

Change backup option in the be-splunkfwd backend respectively for server locate in zoneB.

Of course it would work just fine with ELB, but on the other side local haproxy would give much better control of traffic. Nice to have it for a high logs throughput from host.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...