Bandwidth usage for 200 GB of data from heavy forwarder to indexers

[pranitprakash]

Hello,
I understand from some of the links that using UFs as intermediate forwarding layer add metadata at stream level while using HFs as intermediate layer add metadata at event level.

What is the general increase of daily log transmission size, (say when we are expecting 200 GB of daily data) when passing through this on-premise intermediate layer to Splunk Indexers hosted on an external cloud?

Please advice.

[lfedak_splunk]

Hey @pranitprakash, if they answered your question don't forget to "accept" the answer to award karma points/ close the post.

[gjanders]

The blog post Universal or Heavy, that is the question? shows you the bandwidth usage difference between universal and heavy forwarders.

If you use SSL the data is compressed with an approx 10 to 1 ratio, so there will be a drastic difference with SSL on vs SSL off.

The general answer is it depends, however you can refer to this answer about minimum bandwidth or this answer about measuring bandwidth used.

The 200GB/day doesn't mean that much in terms of bandwidth usage, if you had a perfect distribution that would be just over 8GB/hour + overheads or approx 2400 kb/s, and if you multiply that by 8 it would be kilobits per second, however if you use SSL and you take into account the overheads this number will obviously change.
Also there is a good chance you will have more data at particular times (which you can control by throttling the forwarder as per this answer)...