Getting Data In

Is it possible that the Splunk forwarder caches old events and resends the data again and again?

sanjay_shrestha
Contributor

We have a batch script which monitors files under some folder and then creates a log file with the file name and file creation time information every 5 mins. Currently, that batch script is creating an empty log file as there are no files in the inspected folder, but there are events coming from that log file to Splunk every 5 minutes.

Is it possible that the Splunk forwarder caches old events and re-sends again and again?

0 Karma

fdi01
Motivator

After sending a data block, the forwarder maintains a copy of the data in its wait queue until it receives an acknowledgment. In the meantime, it continues to send additional blocks as usual. If the forwarder doesn't get acknowledgment for a block within 300 seconds (by default), it closes the connection. You can change the wait time by setting the readTimeout attribute in outputs.conf.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

That sounds unlikely as long as indexers are working fine. What's the source field of the repeated data? Can you post the input configuration for that source?

If your indexers are not working fine and you use useAck then there is a small chance of duplicates: http://docs.splunk.com/Documentation/Splunk/6.2.2/Forwarding/Protectagainstlossofin-flightdata#The_p...
...not regularly every five minutes though, and this would be logged in the forwarder's splunkd.log.

0 Karma

sanjay_shrestha
Contributor

It started at noon and Splunk was getting until 12:55 PM however it stopped after 1 PM. Does this mean anything?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...