Getting Data In

Is it possible that the Splunk forwarder caches old events and resends the data again and again?

sanjay_shrestha
Contributor

We have a batch script which monitors files under some folder and then creates a log file with the file name and file creation time information every 5 mins. Currently, that batch script is creating an empty log file as there are no files in the inspected folder, but there are events coming from that log file to Splunk every 5 minutes.

Is it possible that the Splunk forwarder caches old events and re-sends again and again?

0 Karma

fdi01
Motivator

After sending a data block, the forwarder maintains a copy of the data in its wait queue until it receives an acknowledgment. In the meantime, it continues to send additional blocks as usual. If the forwarder doesn't get acknowledgment for a block within 300 seconds (by default), it closes the connection. You can change the wait time by setting the readTimeout attribute in outputs.conf.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

That sounds unlikely as long as indexers are working fine. What's the source field of the repeated data? Can you post the input configuration for that source?

If your indexers are not working fine and you use useAck then there is a small chance of duplicates: http://docs.splunk.com/Documentation/Splunk/6.2.2/Forwarding/Protectagainstlossofin-flightdata#The_p...
...not regularly every five minutes though, and this would be logged in the forwarder's splunkd.log.

0 Karma

sanjay_shrestha
Contributor

It started at noon and Splunk was getting until 12:55 PM however it stopped after 1 PM. Does this mean anything?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...