Splunk Search

Use Regex to extract the fields

siddharthmis
Explorer

I have data like-

2017-12-19 09:39:41|INFO|4b483c4b138de23b2f83a208c2313c4a|8de3f071aed6401d9ff5c4289694e852|a|b|c
2017-12-19 09:39:41|ERRO|4b483c4b138de23b2f83a208c2313c4a|8de3f071aed6401d9ff5c4289694e852|a|b|c|d

Since there are different number of fields, I am not able to extract these.

^(?<timestamp>[^|]+)\|(?<severity>[^|]+)\|(?<x>[^|]+)\|(?<y>[^|]+)\|(?<a>[^|]+)\|(?<b>[^|]+)\|(?<c>[^|^$]+)\|(?<d>.*) works for one but not the other.
Is there any way I can extract all fields.

0 Karma
1 Solution

horsefez
SplunkTrust
SplunkTrust

Hey,

look at this:

https://regex101.com/r/reSnrj/1/
(if it doesn't load properly click in the field with the log events first, this will fix it)

^(?:(?<timestamp>[^|]+)\|(?<severity>[^|]+)\|(?<x>[^|]+)\|(?<y>[^|]+)\|(?<a>[^|]+)\|(?<b>[^|]+)\|(?<c>[^|]+)(?:\|(?<d>[^|]+))?)$

View solution in original post

micahkemp
Champion

Assuming your last values (a, b, c...) are of an unknown quantity, you could do something like this:

props.conf:

[<sourcetype>]
REPORT-sourcetypevalues = severityxy,othervalues

transforms.conf:

[severityxy]
REGEX = ^(?<timestamp>[^|]+)\|(?<severity>[^|]+)\|(?<x>[^|]+)\|(?<y>[^|]+)\|(?<othervalues>.*)$

[othervalues]
SOURCE_KEY = othervalues
REGEX = (^|\|)(?<othervalue>[^|]+)($|\|)
MV_ADD = true
0 Karma

horsefez
SplunkTrust
SplunkTrust

Hey,

look at this:

https://regex101.com/r/reSnrj/1/
(if it doesn't load properly click in the field with the log events first, this will fix it)

^(?:(?<timestamp>[^|]+)\|(?<severity>[^|]+)\|(?<x>[^|]+)\|(?<y>[^|]+)\|(?<a>[^|]+)\|(?<b>[^|]+)\|(?<c>[^|]+)(?:\|(?<d>[^|]+))?)$

cmerriman
Super Champion

This isn't 100% there yet, still noodling, but it's something to start with:

^(?<timestamp>[^|]+)\|(?<severity>[^|]+)\|(?<x>[^|]+)\|(?<y>[^|]+)\|(?<a>[^|]+)\|(?<b>[^|]+)\|(?<c>[^|]+)?[\|]?(?<d>[^|].*)

this should extract them all, but it seems to but c inside d for the event that doesn't have d

0 Karma

Elsurion
Communicator

you can use the same extraction but if the field d is optional you have to define it as optional

(\|(?<d>.*))?

then you can use this regex for both cases.

If you have more optional fields, then you can repeat the optional extraction.

0 Karma

siddharthmis
Explorer

Can someone assist on this?

0 Karma

micahkemp
Champion

Is it always severity, x, y, then a variable number of pipe delimited values afterwards that should all be in one potentially multivalue) field?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...