Splunk Search

Custom parsing

yhemaraj
Engager

I am rookie here.
I have a log of type
"2e 00000008 M 2050 nodemgr 09/10/21 20:01:11.860361 NODEMGR: Successfully set our time"
I would like to extract the fields as below.
deviceId moduleId level id moduleName time(YY,mm,DD HH:MM:SS) message
I do not want to parse the message at this point, but may want to parse a subset of structured messages at a later point.
How do I go about doing this?

Tags (2)
0 Karma

emiller42
Motivator

So two things you'll want to do:

First, create a stanza in transforms.conf that uses regex to parse out your fields. (Below is an example, which is based on the one line you posted. It may need to be tweaked)

[sourcetype_extraction]
REGEX = (\w+)\s+(\d+)\s+(\w+)\s+(\d+)\s+(\w+)\s+(\d+/\d+/\d+\s\d+:\d+:\d+\.\d+)\s(.*)
FORMAT = deviceId::$1 moduleId::$2 level::$3 id::$4 moduleName::$5 time::$6 message::$7

Then you'll want to create a stanza in your props.conf that applies the transform to your sourcetype.

[sourcetype]
REPORT-sourcetype = sourcetype_extraction

All of this is applied at search time, so will apply to anything you've already indexed, and can be changed without losing anything.

It's also worth noting that the timestamp should be getting extracted on index into the _time field, so you shouldn't have to explicitly pull it out. But it may be a good idea to do so anyway via the TIMEFORMAT setting in props.conf.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...