Getting Data In

Issue with log rotation

maniu1609
Path Finder

We have log files that are being monitored. Log files are deleted every 1 hour. We noticed that at the time of log rotation happens, some of the events are missed to indexed in splunk. How can I fix issue so that we don’t miss any data.

0 Karma

micahkemp
Champion

What does your monitor stanza look like, and how are you rotating (filename, compression, etc)?

0 Karma

maniu1609
Path Finder

we have log file aaa.log and if the log file is older than 1hr, then file will be deleted. File monitor stanza just has index, sourcetype and host_regex details alone.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi maniu1609,
the easiest way (if possible) should be to rotate logs in a different file so you can read the last logs in the new file, after a few time (1 or 2 minutes) you can delete it, because the problem is that logs between the last Splunk ingestion and deletion (max 30 seconds) are missed.
If it isn't possible you can reduce Forwarder read interval but anyway you lose something.

Bye.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...