Splunk Search

How to use OR in regex to capture error messages from two different patterns of log files?

vineetc
Engager

So I have the following log structure:

Oct  7 13:51:05, 10.96.3.29, 10.96.3.29, domain:,default [xyz][0x80e003aa][xmlparse][error] mpgw(AbcService): trans(2389130247)[request]: mismatched tag, expected meta at offset 675 of http://xyz/sales/zyx
Oct  7 13:51:05, 10.96.3.28, 10.96.3.28, domain:,default [abc][0x80e00161][mpgw][error] source-http(XyzService): trans(363541717)[x.x.x.x]: Request processing failed: Connection terminated before request headers read because of the connection error occurs, from URL: x.x.x.x:60510

I am trying to get the error_msg in a variable, but since the pattern is not distinct, I have to OR the matching pattern which matches anything after [request]: as Error and anything after [X.X.X.X]: as Error.

I created a regex which work well on a regex tester, but Splunk gives an error:

Error in 'rex' command: Encountered the following error while compiling the regex 
'\[error\][^\(]+\((?<service>[^\)]+).+(trans\((\d+){8}\)\[\b(?:\d{1,3}\.){3}\d{1,3}\b\]\:\s+(?<error_msg>.+) | trans\((\d+){8}\)\[request\]\:\s+(?<error_msg>.+))': Regex: two named subpatterns have the same name

Below is my regex:

\[error\][^\(]+\((?<service>[^\)]+).+(trans\((\d+){8}\)\[\b(?:\d{1,3}\.){3}\d{1,3}\b\]\:\s+(?<error_msg>.+) | trans\((\d+){8}\)\[request\]\:\s+(?<error_msg>.+))

How to capture the ErrorMessage from the two pattern of log files ?
Extracting the fields doesn't work either and gives a lot of junk, so I would like to go through the regex way.

0 Karma

vincenteous
Communicator

Hi vineetc,

Try this regex pattern:

\[error\].+?:\strans\(\d+\)\[(?:request|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\]:\s(?<error_msg>.+)
0 Karma

woodcock
Esteemed Legend

Like this:

 Your Base Search | rex  "\[error\][^\(]+\((?<service>[^\)]+).+(trans\((\d+){8}\)\[\b(?:\d{1,3}\.){3}\d{1,3}\b\]\:\s+(?<error_msg1>.+) | trans\((\d+){8}\)\[request\]\:\s+(?<error_msg2>.+))" | eval error_msg = coalesce(error_msg1, error_msg2)
0 Karma

masonmorales
Influencer

There is a way to make your regex more efficient/cleaner but the quickest answer is:

 \[error\][^\(]+\((?<service>[^\)]+).+(trans\((\d+){8}\)\[\b(?:\d{1,3}\.){3}\d{1,3}\b\]\:\s+|trans\((\d+){8}\)\[request\]\:\s+)(?<error_msg>.+)

Use a pipe (|) within a capture group as an OR operator. You only specify (?.+) once.

0 Karma

masonmorales
Influencer

I have not tested this but just eye balling it, looks like you could get the error message with something like:

trans\(\d+\)\[[^\]+\]\:\s+(?<error_msg>.+)
0 Karma

vineetc
Engager

I was able to add a character(?J) before the variable but seems like I still do get a lot of NULL in error_msg so something is there is my log which still doesnt match

0 Karma

masonmorales
Influencer

What about:

\]\:\s(?<error_msg>.+)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...