Hi,
We have a problem with parsing logic which generates hundreds of thousands of truncate warnings every hour:
06-21-2014 18:14:34.121 -0700 WARN LineBreakingProcessor - Truncating line because limit of 800 has been exceeded with a line length >= 6985 - data_source="/
this issue causes Indexers to choke and eventually create indexing lag.
can someone clarify how to disable indexing of LineBreakingProcessor Warnings in _internal index?
i have tried updating /app/share/splunk/etc/log.cfg category.TailingProcessor=INFO to ERROR but it didnt work.
thanks,
You could explicitly tell Splunk to truncate the line, and then Splunk wouldn't warn you that it did...
Put the following in props.conf
[source::/path/to/the/datasource]
# set the following to the maximum line length that you want
TRUNCATE = 800
Also, the TailingProcessor does not do line-breaking, so you might as well set that back to INFO. If you want to change the setting for the LineBreakingProcessor, edit the file named log-debug.cfg
. You will need to uncomment the line.