Getting Data In

Can you help me figure out why some files were not sent to the Heavy forwarder?

kinaba_splunk
Splunk Employee
Splunk Employee

The universal forwarder (UF) seems to read the following files, but the files were not sent to the heavy forwarder (HF) around 11-26-2018 16:16. The following messages appeared in UF's splunkd.log around that time. It seems that Splunk read the files.

11-26-2018 16:16:16.109 +0900 INFO TailReader - Batch input finished reading file='/fxxxx/splunk/MMM2018092615.txt 
11-26-2018 16:16:17.882 +0900 INFO TailReader - Batch input finished reading file='/fxxxx/splunk/OOO2018092615.txt

Following messages appear in UF's splunkd.log around the same time.

11-26-2018 16:16:05.005 +0900 INFO TcpOutputProc - Queue for group xxxxx_fwd_intermediate has stopped dropping events 11-26-2018 16:16:10.004 +0900 INFO TailReader - Could not send data to output queue (parsingQueue), retrying... 
11-26-2018 16:16:10.005 +0900 WARN TcpOutputProc - Queue for group xxxxx_fwd_intermediate has begun dropping events 

Could you tell me about solution?

0 Karma

kinaba_splunk
Splunk Employee
Splunk Employee

Please check if the file size is large. For example, the size is 20-180MB, it seems that the default queue size of 500KB is really low for them. So, the queue got full is an expected behavior and increasing the queue size should be a solution for that.

UF's outputs.conf 
[tcpout:xxxxx_fwd_intermediate] 
maxQueueSize = 128MB 

HF's inputs.conf 
[splunktcp://9997] 
disabled = 0 
queueSize = 128MB
0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...