Splunk Search

Mapping fields and values using regex and transforms.conf

hlarimer
Communicator

I have a very ugly log file that I need to run a regex against and have it match as many times as possible to map the field name and the value of the field. I have a working regex that I can test through search and its working as expected but when I try to move that regex to a transforms file using REPORT, I'm not seeing any results in my searches. Here are the props.conf and transforms.conf that is running on the SH:

props.conf
[jamfChangeManagement]
REPORT-jamfcm=jamfcm

transforms.conf
[jamfcm]
REGEX=(<_KEY_1>.​*?)\W{5,30}(<_VAL_1>\w.*​)

Any ideas why its not matching the regex in search?

0 Karma
1 Solution

acharlieh
Influencer

Playing with some data you shared in slack and putting it into regex101.com I came up with this for a transforms.conf expression:

[jamfcm]
REGEX = (?m)^\s++(?<_KEY_1>(?:[\w-\/]| (?![ .]))+) [. ]+ (?<_VAL_1>[^\s].*?|)$

Some other questions that you confirmed, the sourcetype of your data is exactly jamfChangeManagement and you restarted after saving changes. You weren't sure if you were running in Fast Mode or not (which obviously would not auto extract fields unless needed).

I then went to try your REGEX from here, and as I pasted into my editor I noticed that copying from here it looks like you have two ZERO WIDTH SPACE (U+200B) characters in your regex which are probably not intended and likely causing havoc as well. (One is before the first asterisk and one after the second)

View solution in original post

acharlieh
Influencer

Playing with some data you shared in slack and putting it into regex101.com I came up with this for a transforms.conf expression:

[jamfcm]
REGEX = (?m)^\s++(?<_KEY_1>(?:[\w-\/]| (?![ .]))+) [. ]+ (?<_VAL_1>[^\s].*?|)$

Some other questions that you confirmed, the sourcetype of your data is exactly jamfChangeManagement and you restarted after saving changes. You weren't sure if you were running in Fast Mode or not (which obviously would not auto extract fields unless needed).

I then went to try your REGEX from here, and as I pasted into my editor I noticed that copying from here it looks like you have two ZERO WIDTH SPACE (U+200B) characters in your regex which are probably not intended and likely causing havoc as well. (One is before the first asterisk and one after the second)

hlarimer
Communicator

I confirmed that I was running in Verbose mode and still didn't get the extractions. I also changed the regex to match what you have above and still don't see any results. I think I'm going to try a very simple regex to confirm that the REPORT function is working correctly.

0 Karma

hlarimer
Communicator

This regex is working, thanks @acharlieh!

0 Karma

acharlieh
Influencer

Other thoughts, could it be a permissions or configuration overlaying issue?

0 Karma

hlarimer
Communicator

Shouldn't be, I'm running as admin.

0 Karma

acharlieh
Influencer

But even as admin, app context can matter.

0 Karma

hlarimer
Communicator

I gotcha, I could set global permissions in local.meta just to be sure.

0 Karma

Yasaswy
Contributor

Hi... just curious, does it work with / did you try using "EXTRACT" option?

0 Karma

hlarimer
Communicator

I have not tried the Extract option, I haven't ever used that command but it may be an option for testing.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...