I need to monitor/splunk multiple files in a directory "/app/products/xxxx/yyy/logs/"
This directory has files like aaa-queries.log.yyyymmdd AND aaa-engineInfo.log.yyyymmdd .
The issue I have is thesefiles will stay unzipped for 3 days ..so today I will have following 3 files for onelog
aaa-queries.log.20111128
aaa-queries.log.20111127
aaa-queries.log.20111126
My inputs.conf and props.conf entries on app/ agent box are as follows
[monitor:///app/products/xxx/yyy/logs/]
index=xxxx
disabled = false
followTail = 1
blacklist = .(gz|zip|bkz|arch|etc)$
[root@ local]# more props.conf
[source::.../aaa-queries.log.d+]
sourcetype = queries_log
[source::.../aaa-engineInfo.log.d+]
sourcetype = engineInfo_log
Becasue I get multiple logs i.e all 3 logs for each source as a part of my source regex ..My sourcetype is not being set properly i.e Is shows as "suggestions-queries.log-too_small" etc . I think because my sourcetype is not set properly I am not able to do field extractions on my search head servers . Is there a way to uniqely set sourcetype in this case . Zipping the old file / changing application is not a option as the logs are huge and other app components need those files to stay for 3 days .
I am wondering if the "d+" is causing the problem. What happens if you change the source stanza in the props.conf to the following:
[source::.../aaa-queries.log...]
sourcetype = queries_log
source::.../aaa-engineInfo.log...]
sourcetype = engineInfo_log
The trailing "..." is a catchall that you can use your configuration files. Here are links to more information:
http://docs.splunk.com/Documentation/Splunk/4.2.4/Data/Specifyinputpathswithwildcards
http://docs.splunk.com/Documentation/Splunk/4.2.4/admin/Propsconf
Thank you ! that seems to work ..If you do not mind what is the trailing "..." doing ? I added d+ cos each log had a date at the end
I am wondering if the "d+" is causing the problem. What happens if you change the source stanza in the props.conf to the following:
[source::.../aaa-queries.log...]
sourcetype = queries_log
source::.../aaa-engineInfo.log...]
sourcetype = engineInfo_log