Hi,
I have a request to monitor a directory, with dynamic logfiles. Sometimes they are there, sometimes the customer will create new ones.... They have validated that the logfiles all follow the same format. Is it possible for splunk to create a sourcetype based upon the name of the file? They all follow the format SERVICE_PID.log. I could create one based upon the directory name, but they would rather use separate sourcetypes, if possible, for easier analysis and reporting.
Hi a212830,
Yes it can 😉 You will need to specify this in your props/transforms files any where parsing/indexing is being performed.
props.conf
[source::...regex_to_match_filename]
TRANSFORMS-fs = force-sourcetype-st
transforms.conf
[force-sourcetype-st]
DEST_KEY = MetaData::Sourcetype
SOURCE_KEY = MetaData::Source
REGEX = YOUR_REGEX_TO_PULL_THE_FILENAME
FORMAT = sourcetype::$1
WRITE_META = true
Hope this helps ...
cheers, MuS
Source type is set at index time so do not think this is possible. I had a similar issue and I used the same source type for everything but added a new filed based on the file source called 'logname'. This is a search time approach and has been working fine.
You could use a transform at index time to add the 'logname' as metadata if needed.
I was wrong you can do this with an index time transform. I always stay away from doing that, i never want to change the raw data,
http://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides
Good one! Knowing his weakness, is a way to strength .... or something like that ..... Yoda would said have 🙂
Hi a212830,
Yes it can 😉 You will need to specify this in your props/transforms files any where parsing/indexing is being performed.
props.conf
[source::...regex_to_match_filename]
TRANSFORMS-fs = force-sourcetype-st
transforms.conf
[force-sourcetype-st]
DEST_KEY = MetaData::Sourcetype
SOURCE_KEY = MetaData::Source
REGEX = YOUR_REGEX_TO_PULL_THE_FILENAME
FORMAT = sourcetype::$1
WRITE_META = true
Hope this helps ...
cheers, MuS
Good stuff, both useful. I'll try them out.