Creating new field on each regex match

[llacoste]

Hello Splunk Community!

As I am quite new to Splunk/Regex, I've got a silly question that may be simple for you:

I have an XML log file with several tags. Those tags can be exactly the same but have different values:

<?xml version="1.0" encoding="UTF-8"?>
<LogName> value1 </LogName>
<EventInLog> Event in the value blabla </EventInLog>
< LogName> value2 </LogName>
<EventInLog> Event in the value blabla </EventInLog>

...
...

And so on..

The thing is I don't know how many of these fields I can have and need to be extracted... I would have gone for a match on something like matching the first one, give the field1 name and so on... but it means I need to define a certain number. What if there is more or less tags?

And each XML is one event. I thought of using line break and breaking after each repetition so I only have an event with one which would solve the problem... but it means If I need to investigate at a precise time on a particular event, I won't be able to have only one event with all the log files involved (I don't know if I'm clear here).

Or maybe in that case I should use 'transactions' to match all the events I just broke into one unique transaction so it would solve this problem?

Maybe do you guys know this problem already and know how to tell splunk to create a new field for each matching tag it finds and gives it a random name?

Thanks for your time and help, I hope I was clear enough, don't hesitate for more info if needed.

Cheers

[patelam2020]

I have create a Regex to Extract Hostname and replace the IP.

My Regex is ---- ^[^ \n]* ([^ ]+)

[sundareshr · Accepted solution]

I would ingest the entire file as a single event and define appropriate mv field extractions. LIke this

props.conf

[unique_stanza]
REPORT-extract_fields = extract_mv_fields

transforms.conf

[extract_mv_fields]
REGEX=LogName\>\s([^\<]+)[\s\S]+?EventInLog\>\s([^\<]+)
FORMAT=$1::$2
MV_ADD=TRUE

Once you have the data in, in your SPL you can break the them apart into separate events for processing using the mvzip/mvexpand command. Like this

base search | streamstats count AS SrNo | eval z=mvzip(LogName, EventInLog, "^") | mvexpand z | rex field=z "(?<LogName>[^\^]+)\^(?<EventInLog>.*)" | fields - z | stats count by SrNo

[llacoste]

Hi,

This might work indeed, I'll try this ASAP!

I completely forgot about mv field extractions and mvexpand...

The road to the knowledge is long ahah 🙂

Thanks for your help,

Cheers