Deployment Architecture

File growth rate must be higher than indexing or forwarding rate.

splunkranger
Path Finder

03-19-2014 18:47:22.017 -0500 INFO BatchReader - Will retry path="\servername1\dnsdebug\dns.log" after deferring for 10000ms, initCRC changed after being queued (before=0xc3e428fff4486a47, after=0x279979c853e7447a). File growth rate must be higher than indexing or forwarding rate.

I am not able to find any information on this error, can anyone comment?

Tags (2)

dmaislin_splunk
Splunk Employee
Splunk Employee

Might this help?

In $SPLUNK_HOME/etc/apps/SplunkUniversalForwarder/local/limits.conf

[thruput]
maxKBps = 0 # means unlimited

dmaislin_splunk
Splunk Employee
Splunk Employee

Please check my answer and accept it. Thanks!

0 Karma

u788332
New Member

I'm seeing same issue but for only one sourcetype , where ~ 10 sourcetypes are specified. I get this for the one sourcetype "File growth rate must be higher than indexing or forwarding rate" , limits.conf is set to 512kbps and is working well for other sourcetypes. The result is a large gaps in indexing for this specific log. However this log is very active many lines per second. Can a bottleneck occur only for high written logs? you would think it would happen on all sourcetypes at the same point in time.. thanks

0 Karma

dmaislin_splunk
Splunk Employee
Splunk Employee

You can get creative and run another forwarder on different ports on the same system when there is a scenario as you described. It helps eliminate that bottleneck where you can dedicate that forwarder to just that one heavy sourcetype and adjust that specific limits.conf for that.

0 Karma

splunkranger
Path Finder

That did solve the issue, just surprised there was no more information available on this error.
Thank you!

0 Karma

somesoni2
SplunkTrust
SplunkTrust

It seems before Splunk can read the file, its getting changed, especially first few character which Splunk uses for CRC check. Does the program generates the log file overwrites the log file?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...