Hi Splunkers!
Beyond configuring the autodetection of new sourcetypes, is specifying sourcetype detection via a wildcard possible when the log file is within a zip file?
For the sake of illustration, I have a single zip file, with the following two files:
/data/db2.zip
- AUDIT_AUTHORIZATION_FAILURES.log
- AUDIT_FIRST_READ.log
Example:
[batch:///data/db2.zip:AUDIT_AUTHORIZATION_FAILURES*]
index = db2
sourcetype = db2_audit_auth_fail
move_policy = sinkhole
queue = parsingQueue
[batch:///data/db2.zip:AUDIT_FIRST_READ*]
index = db2
sourcetype = db2_audit_first_read
move_policy = sinkhole
queue = parsingQueue
Not pretty I know, but I'd rather avoid having to configure automatic sourcetype recognition as the events are multi-lined and painful.
Cheers & Beers,
RT
So I've found the way to fix the issue. Luckily (as Drainy asked) there are only a few distinct filenames in each of the archives, such as:
db2_20120620.zip
- AUDIT_FIRST_READ.TXT
- AUDIT_FIRST_WRITE.TXT
- AUDIT_AUTHORIZATION_FAIL.TXT
So to apply file specific sourcetypeing I did the following:
- In my inputs.conf, I don't specify the sourcetype.
- In my props.conf, I override the sourcetype based on the source. eg:
[source::.../db2_\d+.zip:./AUDIT_FIRST_READ.TXT]
sourcetype = db2_audit_first_read
[source::.../db2_\d+.zip:./AUDIT_FIRST_WRITE.TXT]
sourcetype = db2_audit_first_write
[source::.../db2_\d+.zip:./AUDIT_AUTHORIZATION_FAIL.TXT]
sourcetype = db2_audit_authorization_fail
Then I define my sourcetypes as I would normally.
Hope this helps someone 🙂
They are indeed. Thanks for the comment... it twigged how I should do it. Thankfully there are only a few different types of files and they are consistently named.
So I've found the way to fix the issue. Luckily (as Drainy asked) there are only a few distinct filenames in each of the archives, such as:
db2_20120620.zip
- AUDIT_FIRST_READ.TXT
- AUDIT_FIRST_WRITE.TXT
- AUDIT_AUTHORIZATION_FAIL.TXT
So to apply file specific sourcetypeing I did the following:
- In my inputs.conf, I don't specify the sourcetype.
- In my props.conf, I override the sourcetype based on the source. eg:
[source::.../db2_\d+.zip:./AUDIT_FIRST_READ.TXT]
sourcetype = db2_audit_first_read
[source::.../db2_\d+.zip:./AUDIT_FIRST_WRITE.TXT]
sourcetype = db2_audit_first_write
[source::.../db2_\d+.zip:./AUDIT_AUTHORIZATION_FAIL.TXT]
sourcetype = db2_audit_authorization_fail
Then I define my sourcetypes as I would normally.
Hope this helps someone 🙂
Are the events within the two log files quite distinct?