All,
I have a dozen+ inputs I am creating. I feel there there should be a smarter way of doing this. As you can see, I am naming the sourcetype after the log.
[monitor:///opt/paidsearch/autopilot/logs/collateral.log]
index=paidsearch
sourcetype=paidsearch:collateral
[monitor:///opt/paidsearch/autopilot/logs/partner.log]
index=paidsearch
sourcetype=paidsearch:partner
inputs.conf
[monitor:///opt/paidsearch/autopilot/logs]
index=paidsearch
sourcetype=paidsearch
props.conf (at the same Splunk instance because this is input stage configuration)
[source::/opt/paidsearch/autopilot/logs/collateral.log]
sourcetype=paidsearch:collateral
[source::/opt/paidsearch/autopilot/logs/partner.log]
sourcetype=paidsearch:partner
There must be all kinds of ways - I create tiny little Java programs for this type of cases...
Yes there is.
(NOTE I HAVEN'T TESTED THE BELOW)
Collection layer (normally a universal forwarder):
[monitor:///opt/paidsearch/autopilot/logs/*.log]
index=paidsearch
sourcetype=paidsearch:rename
Parsing layer (before indexing, usually a heavy forwarder or indexer):
# props.conf
[paidsearch:rename]
TRANSFORMS-changesourcetype = set_paidsearch_sourcetype_from_filename
# transforms.conf
[set_paidsearch_sourcetype_from_filename]
SOURCE_KEY = MetaData::Source
REGEX = ([^\/]+)\.\w+$
FORMAT = paidsearch:$1
DEST_KEY = MetaData::Sourcetype
WRITE_META = true
I usually think changing sourcetype is evil and should be avoided. But this is nice. I like it.