Splunk Search

rex - multiple matches and using |

jshaynes
Explorer

I'm trying to run several field extractions using the rex command. Here is a sample log format:

ironportmail: Info: MID 42342 ICID 1234 From: xyz@yyx.com
ironportmail: Info: MID 42342 ICID 1234 To: abc@def.com

I'd like to extract the MID, ICID, From and To fields using just one rex command. (This is so I can wrap an eventtype around it.)

Conceptually I'm looking for something like this, but havent been able to get it to work:

rex field=_raw "MID (?[^ ]+) | ICID (?[^ ]+) | From: <(?[^>]+)> | To: <(?[^>]+)"

Tags (1)
0 Karma
1 Solution

Ayn
Legend

I think you'd be better off putting this in props.conf / transforms.conf instead so you won't have to specify your field extractions explicitly in each search that you want to use these fields.

That said, this will work, though with a caveat:

... | rex field=_raw max_match=10 "(?:MID (?<mid>[^ ]+)|ICID (?<icid>[^ ]+)|From: <(?<mailfrom>[^>]+)>|To: <(?<mailto>[^>]+))"

The caveat being that you will need to provide rex with a max_match (default is 1), otherwise it will stop once it finds its first match.

View solution in original post

Ayn
Legend

I think you'd be better off putting this in props.conf / transforms.conf instead so you won't have to specify your field extractions explicitly in each search that you want to use these fields.

That said, this will work, though with a caveat:

... | rex field=_raw max_match=10 "(?:MID (?<mid>[^ ]+)|ICID (?<icid>[^ ]+)|From: <(?<mailfrom>[^>]+)>|To: <(?<mailto>[^>]+))"

The caveat being that you will need to provide rex with a max_match (default is 1), otherwise it will stop once it finds its first match.

jshaynes
Explorer

couple of follow up questions:

  1. what does the ?: at the beginning do. the query seems to work fine without it
  2. any way to prevent all of the empty matches that get returned as a result of changing max_match?

ex, I get a bunch of empty mailfrom, mailto, etc. fields since they don't all occur on every line:

sample field list from search results:
mailfrom="" | mailfrom="" | mailfrom="" | mailfrom=""abc@bvd.com" | mailto="" | mailto="" | mailto="xyz@abc.com" | ... etc

0 Karma

jshaynes
Explorer

awesome, that did it. thanks.

the reason im running these inline is because I am searching off of a summary index. those fields do already get extracted automatically for regular searches, but since the fields get dropped after doing a " | collect index=summary", im having to recreate them.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...