All Apps and Add-ons

How to extract fields from the message of a windows event log?

john0499
Explorer

Hi all,

I have a problem extracting fields after the Message= portion of a windows application event log. Everything after Message= is being added as single value to the message field.

I think it may be the due to the formatting of the data after Message=, which seems to have a leading space or tab. It looks like this:

Message=Process started in passive mode.

Command Line: C:\something\something\something.exe
Process Id: 8988
...

(the the white space is being removed from this post - its in front of 'Command' and 'Process')

There are about 50 more fields after these, all on new lines but with a space or tab before them.

I have tried playing with the regex from transforms.conf below, but I've only been able to move the problem from the message field to a new field called Command_Line that contains a value of everything below it.

[wel-col-kv]
SOURCE_KEY = Message
REGEX = \n([^:\n\r]+):[ \t]++([^\n]*)
FORMAT = $1::$2
MV_ADD = true

I have found some similar questions asked but haven't been able to come up with a solution yet. Any help would be greatly appreciated.

0 Karma
1 Solution

john0499
Explorer

I do have the TA, although I don't believe it's actually needed as windows event extraction is included in the default transforms.conf

I managed to get it working with the following changes to the default:

REGEX = \n? \t:[ \t]++([^\r]*)

FYI this was to extract avecto defendpoint events

View solution in original post

0 Karma

john0499
Explorer

I do have the TA, although I don't believe it's actually needed as windows event extraction is included in the default transforms.conf

I managed to get it working with the following changes to the default:

REGEX = \n? \t:[ \t]++([^\r]*)

FYI this was to extract avecto defendpoint events

0 Karma

jwindley_splunk
Splunk Employee
Splunk Employee

Hi John, I have the same problem parsing the avecto logs from Message. I tried your new regex but it didn't solve it for me. Strangely though the default regex parses everything fine for me in regex101. Did you make any other changes at all other than to the wel-col-kv settings?

Thanks

0 Karma

john0499
Explorer

Hi jwindley,

this is what I have in my avecto app:

props.conf
[source::WinEventLog:Application]
REPORT-MESSAGE = avec-wel-message, avec-wel-eq-kv, avec-wel-col-kv
KV_MODE=none

Note the below settings are effectively legacy, in place here to handle

data coming from much much older forwarders (3.x & 4.x)

SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD=30
LINE_BREAKER = (\r\n)
TRANSFORMS-FIELDS = strip-winevt-linebreaker

transforms.conf
[avec-wel-message]
REGEX = (?sm)^(?<_pre_msg>.+)\nMessage=(?.+)$
CLEAN_KEYS = false

[avec-wel-eq-kv]
SOURCE_KEY = _pre_msg
DELIMS = "\n","="
MV_ADD = true

[avec-wel-col-kv]
SOURCE_KEY = Message
REGEX = \n? \t:[ \t]++([^\r]*)
FORMAT = $1::$2
MV_ADD = true

0 Karma

adonio
Ultra Champion

are you using the Windows TA?
https://splunkbase.splunk.com/app/742/

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...