Splunk Search

Log file rotation with date in file name

1234testtest
Path Finder

Hi,
I have to get all (and ONLY) tomcat std out files in D:/Program Files/Apache Software Foundation/Tomcat 6.0/logs. I have the following name of tomcat std out file tomcat6-stdout.2013-01-03.log - and it changes everyday. How do I monitor these files continuosly

My inputs.conf has the stanza below.
[monitor://D:/Program Files/Apache Software Foundation/Tomcat 6.0/logs/tomcat6-stdout*]
disabled = false
followTail = 0
sourcetype = w_std_log_dynamic

I suspect the problem is with log file rotation as if a new tomcat6-stdout with today's date is getting generated, it is not being indexed (not shown in the sources list in the search app).

Thanks and any pointers would be of great help.

Kindly help

Tags (1)
0 Karma
1 Solution

alacercogitatus
SplunkTrust
SplunkTrust

You could try using a whitelist for the folder. The * in the stanza relates to path, so your monitor was looking for folders instead of files that match the regex.

[monitor://D:/Program Files/Apache Software Foundation/Tomcat 6.0/logs]
disabled = false
followTail = 0
sourcetype=w_std_log_dynamic
whitelist = tomcat6-stdout.*\.log

View solution in original post

alacercogitatus
SplunkTrust
SplunkTrust

You could try using a whitelist for the folder. The * in the stanza relates to path, so your monitor was looking for folders instead of files that match the regex.

[monitor://D:/Program Files/Apache Software Foundation/Tomcat 6.0/logs]
disabled = false
followTail = 0
sourcetype=w_std_log_dynamic
whitelist = tomcat6-stdout.*\.log

1234testtest
Path Finder

Thank you. surprisingly after a restart of splunk server - this worked -

[monitor://D:\Program Files\Apache Software Foundation\Tomcat 6.0\logs\tomcat6-stdout*]
disabled = false
followTail = 0
sourcetype = w_std_log_dynamic

lukejadamec
Super Champion

This worked for me also. There are many different sourcetypes in this directory, so for this particular sourcetype with the date in the file name is easier to manage as a source with a wildcard than with a whitelist for a directory - I need all files from the directory, but with different sourcetypes.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

The primary reason for not doing it that way is that the assumption was that this was a Windows system "D:" and the documentation states that that will not work on Windows.

Warning: In Windows, you cannot currently use a wildcard at the root level. For example, this does not work:
[monitor://E:\...\foo\*.log]
Splunk logs an error and fails to index the desired files.
This is a known issue, described in the Known Issues topic of the Release Notes. Look there (http://docs.splunk.com/Documentation/Splunk/5.0.1/ReleaseNotes/KnownIssues) for details on all known issues.

0 Karma

kallu
Communicator

How about adding ".log" to the end. Like this

[monitor://D:/Program Files/Apache Software Foundation/Tomcat 6.0/logs/tomcat6-stdout*.log]

This is then exactly like in examples in Splunk docs (=should work)

http://docs.splunk.com/Documentation/Splunk/latest/Data/Specifyinputpathswithwildcards

Ps. I'm not sure if it makes difference but have you tried turning your "/" to "\" if it is Windows?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...