Getting Data In

Splunk 3.4.4 LWF doesn´t process data until logrotate happens. Why?

tpaulsen
Contributor

We have on four Linux SLES10_64 Servers Splunk 3.4.4. Forwarders installed. Usually our production logs produce a constant stream of at least 30 events/minute during the night time. Due to performance issues with these production boxes, we recently switched those Forwarders into LWF mode, to reduce their footprint. Now we found out that during the night time, when the data stream on the logs drop down to app. 30 events/minute, the LWFs don´t forward any data to our Indexer until at 8am the logrotate sets in. During the day time the LWFs work fine until app. 11pm plus minus 2 hours.

Is this a bug in the Splunk 3.4.4. LWF? Or could it be a licensing issue? The logs of these LWFs don´t show anything.

0 Karma

Mick
Splunk Employee
Splunk Employee

It's not a licensing issue, as forwarders don't require a license unless they are indexing data locally.

It sounds like a bug to me, if it was working ok with the regular Forwarder, and then switching to the LWF broke it, that suggests that the LWF isn't checking your files correctly.

It could also be something to do with the way the file is updated by your logging app, maybe it's not updating the modtime during those hours?

I recommend using the troubleshooting instructions here to enable DEBUG settings and figure out what Splunk is seeing when it checks these files.

A last resort would be to add the 'alwaysOpenFile' setting in inputs.conf, but seeing as you want to reduce resource footprint that may not be for you

By the way, 3.4.4 was a short-lived build, if you're planning an upgrade, look into 4.1.1 as that has a much improved file-monitoring capability

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...