How to configure the universal forwarder to Heavy forwarder then to an Indexer?

[pankajupadhyay]

Hi,

Can someone help what are the step I need to do if I have below flow :

Universal Forwarder ------- Heavy forwarder ------- Indexer

And need help how to parse the traffic when the log will at heavy forwarder from Universal Forwarder.

[woodcock]

What do you mean by parse? In any case, all of that kind of thing happens at the first FULL (non-UF) instance of Splunk that handles the events, which in your case should be your HF tier.

As far as sending to HF from UF through a load balancer, THIS IS NOT SUPPORTED however, if you do it correctly, it will work.
1: Turn on sticky sessions on the F5.
2: DO NOT use indexer discovery.
3: Open multiple incoming ports on your indexers to FORCE the LB to switch when the UFs switch. If you are using round-robin (default), then 3-5 will do. Let's say that you use 9997, 9998 and 9999 and have 9 HFs. You should create your outputs.conf like this:

server=HF1:9997, HF2:9998, HF3:9999, HF4:9997, HF5:9998, HF6:9999, HF7:9997, HF8:9998, HF9:9999

If you do not do the multiple port thing, then the F5 will cause your UFs to lock on to an HF basically forever because the F5 will balance based on number of connections, not source/type of data and will prefer to keep/re-establish the current connection when more data comes from the same source. Yes, this defeats the point of the LB but you SHOULD NOT BE USING THEM ANYWAY. I assume that the reason that you are is because your network/security team policies make this the easiest way to get data from source to dest (I get it). The point is: DO NOT let the F5 actually do any load-balancing. You will regret it.

[jacobpevans]

Greetings @pankajupadhyay,

Please read this documentation: https://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf

Of specific interest to you is the part below. You'll need to check and/or update both your UF and HF.

[tcpout:<target_group>]
server = [<ip>|<servername>]:<port>, [<ip>|<servername>]:<port>, ...
* A comma-separated list of one or more systems to send data to over a
  TCP socket.
* Required if the 'indexerDiscovery' setting is not set.
* Typically used to specify receiving Splunk systems, although you can use
  it to send data to non-Splunk systems (see the 'sendCookedData' setting).
* For each system you list, the following information is required:
  * The IP address or server name where one or more systems are listening.
  * The port on which the syslog server is listening.

[pankajupadhyay]

Hi Jacob,

Can you please do let me know how to configure if my environment as mentioned below:

UF----F5 Loadblanacer--------HF

When the UF will send the logs it will go to F5 loabalancer VIP and F5 loadbalancer will forward to HF.

I have defined the F5 VIP IP in outputs.conf on UF but missing data is not coming on HF.

So could you please let me know do I need to define receiving port listener on HF ?