Splunk Search

Multiple regex in a field extraction

byu168
Path Finder

Hi,

What I mean is that I want to parse all the error messages in my logs into one field called Errors but the regular expressions are different.

Examples:
Error: exceed max iterations, iter 120, count_trial 120
ERROR setup_acap_venv.sh failed.
ERROR [ac_analysis.tools.merge_annotations:327]

They don't quite all match up so one field extraction won't encompass all of them. Is there a way to have multiple regex that go into one field? Or is there a way to handle this when indexing the data instead of creating a field extraction?

0 Karma
1 Solution

micahkemp
Champion

Yes, you can definitely have multiple field extractions in to the same field.

props.conf

[<sourcetype>]
REPORT-yourfield = yourfield1,yourfield2,yourfield3

transforms.conf

[yourfield1]
REGEX = (?<yourfield>blahblahblah)

[yourfield2]
REGEX = (?<yourfield>moreblahmoreblah)

[yourfield3]
REGEX = (?<yourfield>evenmoreblah)

View solution in original post

woodcock
Esteemed Legend

Yes, you can do this in the CLI by piping to a series of regex commands back-to-back with the same capture name. The last successful one will win but none of the unsuccessful ones will damage a previously successful field value creation.

0 Karma

jwalbert
New Member

One field extract should work, especially if your logs all lead with 'error' string prefix. Simple extraction based on your sample events:

(?i)error[\s:]+(?.*) OR (?i)error[^\w]+(?.*(?\]|\.))

The first one being the more simple/straightforward of the two, with the latter aiming to clean up the extracted data if you are so inclined.

perl -ne 'print $1.$/ if /error[^\w]+(.*(?<!\]|\.))/i' re_sample
exceed max iterations, iter 120, count_trial 120
setup_acap_venv.sh failed

ac_analysis.tools.merge_annotations:327

Joe

0 Karma

micahkemp
Champion

Yes, you can definitely have multiple field extractions in to the same field.

props.conf

[<sourcetype>]
REPORT-yourfield = yourfield1,yourfield2,yourfield3

transforms.conf

[yourfield1]
REGEX = (?<yourfield>blahblahblah)

[yourfield2]
REGEX = (?<yourfield>moreblahmoreblah)

[yourfield3]
REGEX = (?<yourfield>evenmoreblah)

byu168
Path Finder

Thanks! Worked perfectly

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...