Getting Data In

Parsing Queue blocked on Heavy Forwarder

omuelle1
Communicator

Hey guys,

I got some question regarding parsing queue issues I have been observing on our Heavy Forwarders. I am currently seeing between 500 and 1000 blocked events on each heavy forwarder daily when running:

index=_internal host=HF blocked=true

The total ratio of blocked events seems to be about 10% and they mostly all seem to appear in the aggqueue:

alt text

My main question is if this is reason for concern or what the impact on my current Splunk environment would be. Also why would all this blocking be in mainly one queue ?

Thank you,

Oliver

0 Karma
1 Solution

codebuilder
SplunkTrust
SplunkTrust

This generally indicates that you have not adjusted the thruput setting on your HF from the default of 256kbs.
My suggestion is to change/add the value in limits.conf to maxKBps=0, or a number greater than the default that you think your network can support. The forwarder is being throttled and cannot keep up with the data it's trying to send to the indexers.

[thruput]
maxKBps = (0 = unlimited)

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf

----
An upvote would be appreciated and Accept Solution if it helps!

View solution in original post

0 Karma

codebuilder
SplunkTrust
SplunkTrust

This generally indicates that you have not adjusted the thruput setting on your HF from the default of 256kbs.
My suggestion is to change/add the value in limits.conf to maxKBps=0, or a number greater than the default that you think your network can support. The forwarder is being throttled and cannot keep up with the data it's trying to send to the indexers.

[thruput]
maxKBps = (0 = unlimited)

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

anmolpatel
Builder

The issue appears to be at the aggQueue based on the screenshot.
So check if the props.conf is configured correctly for the sourcetypes.
Things you want to check:
- Should_Linemerge
- Max_events
- Time_prefix
- Time_format
- Datetime_config
- Max_Days_Ago

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...