Getting Data In

Bandwidth usage for 200 GB of data from heavy forwarder to indexers

pranitprakash
Explorer

Hello,
I understand from some of the links that using UFs as intermediate forwarding layer add metadata at stream level while using HFs as intermediate layer add metadata at event level.

What is the general increase of daily log transmission size, (say when we are expecting 200 GB of daily data) when passing through this on-premise intermediate layer to Splunk Indexers hosted on an external cloud?

Please advice.

0 Karma

lfedak_splunk
Splunk Employee
Splunk Employee

Hey @pranitprakash, if they answered your question don't forget to "accept" the answer to award karma points/ close the post.

0 Karma

gjanders
SplunkTrust
SplunkTrust

The blog post Universal or Heavy, that is the question? shows you the bandwidth usage difference between universal and heavy forwarders.

If you use SSL the data is compressed with an approx 10 to 1 ratio, so there will be a drastic difference with SSL on vs SSL off.

The general answer is it depends, however you can refer to this answer about minimum bandwidth or this answer about measuring bandwidth used.

The 200GB/day doesn't mean that much in terms of bandwidth usage, if you had a perfect distribution that would be just over 8GB/hour + overheads or approx 2400 kb/s, and if you multiply that by 8 it would be kilobits per second, however if you use SSL and you take into account the overheads this number will obviously change.
Also there is a good chance you will have more data at particular times (which you can control by throttling the forwarder as per this answer)...

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...