Getting Data In

Is there a maximum number of syslog events that forwader can receive directly?

Takajian
Builder

I will set up AutoLB on intermediate forwarder. The syslog event from many network devices will be sent to the forwarder and it forward the data to multiple indexers. The syslog data size will be over 10 GB per day. If there is a maximum number of syslog events that single forwarder can receive? I understand if I use syslog-ng and monitor the syslog text file, I do not care about this. However I have to use windows environment, so I can not use syslog-ng, the forwarder need to receive syslog directly.

Tags (1)
0 Karma

stuartamurray
Path Finder

Hmm this is not necessarily true.

The indexer can for sure take 10GB per day easily but you may have an issue if your forwarders are trying to push that amount as they are limited. Depends what your peak traffic rate is.

By default a lightweight forwarder is limited to 256KBps or around 15MBph (thats Bytes not bits). So you may find your forwarders getting behind.

I've seen instances where my forwarders are 8 hours behind, although with much chunkier log sizes.

My solution has been a combination of: Either run as a standard rather than lightweight forwarder Comment out the maxKBps directive in limits.conf (has the advantage of running without web) Reduce the log size produced by that node.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

There is no maximum number, and 10GB/day is a very low rate. Of course, if you have extremely high intra-day peaks, the network, network stack, or OS may drop packets.

Get Updates on the Splunk Community!

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...