Override sourcetype for custom logs

[sundarrajan]

Hi I am trying to override my current sourcetype to create multiple source types based on key matching patterns. But the settings are not working, my settings are as follows, pls let know me where I go wrong,
pros. conf
[transaction:logs]
BREAK_ONLY_BEFORE_DATE = true
SHOULD_LINEMERGE = true
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 25
TRANSFORMS - sourcetypeoverwrite =receipts, businesstransaction

transforms.conf
[receipts]
DEST_KEY = MetaData:Sourcetype
REGEX = (%retail)
FORMAT = sourcetype::transaction:logs

[businesstransaction]
DEST_KEY = MetaData:Sourcetype
REGEX = (%transaction)
FORMAT = sourcetype::transaction:logs

I also tried rule:: option but its not working as well in my props.conf
[rule::receipts]
sourcetype = receipt
MORE_THAN_0 = (%retail)

[rule::businesstransaction]
sourcetype = businesstransaction
MORE_THAN_0 = (%transaction)

Yet am not getting results in either of methods. Is there any better way to look into this.

[sundarrajan]

Hi @nickhills thanks for the response. %retail% & %transaction are the key words. I tried to use these key words from the logs as condition to override the event to a new sourcetype.
Please do suggest, if we can use keywords (%retail) in place of regex pattern for the key word. I also tried this key word as part of my rules(MORE_THAN_1 line in the event), but it didn't work.

[nickhills]

try my transforms example from above, I think it should work for you.

[sundarrajan]

@nickhills I tried the transforms.conf, still its not showing up results for me. The input log is XML file and i have other custom conditions like LINE_BREAKER, BREAK_ONLY_BEFORE, MUST_BREAK_AFTER conditions in my props. I presume that is not making any impact on this transforms.conf settings.

[nickhills]

oh so literal %retail is what you mean?

so you are matching something like this:
eventdata=some text from your log blah blah %retail value=23.45
in which case, your regex should be fine 🙂

[sundarrajan]

I also tried with INGEST_EVAL
INGEST_EVAL = sourcetype:=case(sourcetype=="transaction", "businesstransaction", sourcetype=="retail", "receipts", true(), sourcetype), still its not splitting as expected.

[nickhills]

You don't say what you want the new sourcetypes called?

I'll assume they should be called transaction:receipts and transaction: businesstransaction ...
transforms.conf

[receipts]
DEST_KEY = MetaData:Sourcetype
REGEX = (%retail)
FORMAT = sourcetype::transaction:receipts

[businesstransaction]
DEST_KEY = MetaData:Sourcetype
REGEX = (%transaction)
FORMAT = sourcetype::transaction:businesstransaction

Im also assuming you are using (%retail) as a placeholder to refer to some regex which matches in your relevant log? %retail is itself not a valid regex expression. so you might need to use something which actually matches. potentially something like:

eventdata=some text from your log which indicates this is a retail log value=23.45 

regex=\w\s(retail)\s