Keep specific events and discard the rest (PART 2)

[Log_wrangler]

Hi,

I followed previous instructions and successfully was able to keep only ERROR and WARN logs and "discard the rest" from my_app logs that I am monitoring with a UF and filtering on the HF.

Now I need to filter down the ERROR and WARN logs a bit more, e.g. remove any event that contains foo-failure.

Here is what I have (based on previous answer)

Props.conf

[source::/temp/my_app/logs/*]
TRANSFORMS-set=setnull_my_app_logs,setparsing_my_app_logs

Transforms.conf

[setnull_my_app_logs]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing_my_app_logs]
REGEX = (?i)(ERROR|WARN)
DEST_KEY = queue
FORMAT = indexQueue

Do I need to add another setnull stanza in transforms.conf ???

Like
[setnull_my_app_logs]
REGEX = foo-failure
DEST_KEY = queue
FORMAT = nullQueue

or do I need to integrate "foo-failure" into the discard stanza somehow?

Thank you

[somesoni2 · Accepted solution]

See if this works for you

Props.conf

[source::/temp/my_app/logs/*]
TRANSFORMS-set=setnull_my_app_logs,setparsing_error_warn_logs,setnull_foo_error

Transforms.conf

[setnull_my_app_logs]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing_error_warn_logs]
REGEX = (?i)(ERROR|WARN)
DEST_KEY = queue
FORMAT = indexQueue

[setnull_foo_error]
REGEX = foo-failure
DEST_KEY = queue
FORMAT = nullQueue

So,setnull_foo_error marks all events to go to nullQueue, setparsing_error_warn_logs marks all ERROR/WARN events to indexQueue leaving rest to still marked to go to nullQueue, at last, setnull_foo_error marks all ERROR/WARN events which contains foo-failure to mark to go to nullQueue, thus only ERROR/WARN events which DO NOT contains foo-failure will get indexed.

[Log_wrangler]

Thank you!