Getting Data In

Splunk Universal Forwarder Slow

ebay_nkitmitto
Engager

My Splunk forwarder is forwarding ~5 events per second (per my Deployment Monitor on my search head). The maxKBps is set to 20,000 and Splunk is transferring at a rate of 16,000 KBps. Why is the number of events so low? I have 27TB of data (and then some) that need to be indexed, these are historical log files.

I do have indexer acknowledgement enabled in my environment as well.

1 Solution

ebay_nkitmitto
Engager

By adding "maxQueueSize=500MB" to the outputs.conf file, under the [tcpout] stanza.

Change the 500MB to whatever you wish. It defaults to 500kb I believe. But be careful, because whatever you set this number to, it will consume 3x more RAM. So if you have it set to 500MB, it'll consume 1500MB of RAM.

Please this for more info (search for maxQueueSize):
http://docs.splunk.com/Documentation/Splunk/5.0.2/Admin/Outputsconf

View solution in original post

ebay_nkitmitto
Engager

By adding "maxQueueSize=500MB" to the outputs.conf file, under the [tcpout] stanza.

Change the 500MB to whatever you wish. It defaults to 500kb I believe. But be careful, because whatever you set this number to, it will consume 3x more RAM. So if you have it set to 500MB, it'll consume 1500MB of RAM.

Please this for more info (search for maxQueueSize):
http://docs.splunk.com/Documentation/Splunk/5.0.2/Admin/Outputsconf

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...