Getting Data In

logrotate (sort of) file monitoring issue in Splunk LWF 6.0.2

d646800
Explorer

we have multiple files that are being monitored ; file.1, file.2, file.3 Bob.1, Bob.2, Bob.3, Cat.1 Cat.2, Cat3. ...etc

for file.1, all new events will be updated only in file.1 and once it reach a certain quota, it will start updating only file.2 and then only file.3. once file.3 is full and it will start with file.1 again. So are the same for Bob and Cat logs.

in my inputs

[monitor:///path/]

whitelist=(file|Bob|Cat)

index = ABC

recursive = false

ignoreOlderThan = 14d

disabled = false

crcSalt = "<"SOURCE">"

in props

[source::/PATH/file*]

sourcetype = ABC_file

CHECK_METHOD = entire_md5

[source::/PATH/Bob*]

sourcetype = ABC_Bob

CHECK_METHOD = entire_md5

[source::/PATH/Cat*]

sourcetype = ABC_Cat

CHECK_METHOD = entire_md5

and this works great no issue here

The issue is every time there is file change, for example. file.1 being full and events starts coming in to file.2 , Splunk cannot seems to detect it? so are for Bob and Cat file. I have to restart splunk and it will start monitoring file.2 for a while until it move to file.3? am i missing something here?

Tags (2)
0 Karma

lguinn2
Legend

I believe that you should remove the ignoreOlderThan = 14d

Unless the files are all the same at the beginning - you should also remove the crcSalt = "<"SOURCE">"

d646800
Explorer

thanks. once the files are deemed to be older than 14 days it actually get store in the fishbucket and will never be monitored ever again though the files are updated later on and modtime change etc. So yes you are right, that's the problem. thanks for the pointers!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...