Getting Data In

If Universal Forwarder crashes, can we throttle the rate at which it sends data to indexer?

roychen
Path Finder

Hello,

Assuming that I have a universal forwarder configured to monitor a directory of flat files, e.g. /var/log/, what happens if the following sequence of events happens?

  1. Universal forwarder is monitoring files in /var/log
  2. Universal forwarder crashes for some reason, or someone accidentally kills the process
  3. Files in /var/log are modified, written to, etc. Assume a large number of changes have been made
  4. Universal forwarder is restarted

In this situation, will the universal forwarder simply check through /var/log for any modified files, and send all the changes in the logs to the indexer at one go, thus possibly saturating the network bandwidth?

I believe the universal forwarder's max throughput is 256 kb/s, so if there's a large amount of changes, will it always attempt to send data to the indexer at this maximum rate?

Is there any way to throttle the universal forwarder's sending rate?

Tags (1)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

The throttled is set to a 256 Kb/s on a UF, but you can set this to whatever rate you like, higher or lower, in the limits.conf file, e.g.:

[thruput]
maxKBps = 128

Setting it to "0" makes the maximum rate unlimited (up to the capacity of the process and the machine).

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

The throttled is set to a 256 Kb/s on a UF, but you can set this to whatever rate you like, higher or lower, in the limits.conf file, e.g.:

[thruput]
maxKBps = 128

Setting it to "0" makes the maximum rate unlimited (up to the capacity of the process and the machine).

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...