Hello Splunk Community!
As I am quite new to Splunk/Regex, I've got a silly question that may be simple for you:
I have an XML log file with several tags. Those tags can be exactly the same but have different values:
<?xml version="1.0" encoding="UTF-8"?>
<LogName> value1 </LogName>
<EventInLog> Event in the value blabla </EventInLog>
< LogName> value2 </LogName>
<EventInLog> Event in the value blabla </EventInLog>
...
...
And so on..
The thing is I don't know how many of these fields I can have and need to be extracted... I would have gone for a match on something like matching the first one, give the field1 name and so on... but it means I need to define a certain number. What if there is more or less tags?
And each XML is one event. I thought of using line break and breaking after each repetition so I only have an event with one which would solve the problem... but it means If I need to investigate at a precise time on a particular event, I won't be able to have only one event with all the log files involved (I don't know if I'm clear here).
Or maybe in that case I should use 'transactions' to match all the events I just broke into one unique transaction so it would solve this problem?
Maybe do you guys know this problem already and know how to tell splunk to create a new field for each matching tag it finds and gives it a random name?
Thanks for your time and help, I hope I was clear enough, don't hesitate for more info if needed.
Cheers
I would ingest the entire file as a single event and define appropriate mv field extractions. LIke this
props.conf
[unique_stanza]
REPORT-extract_fields = extract_mv_fields
transforms.conf
[extract_mv_fields]
REGEX=LogName\>\s([^\<]+)[\s\S]+?EventInLog\>\s([^\<]+)
FORMAT=$1::$2
MV_ADD=TRUE
Once you have the data in, in your SPL you can break the them apart into separate events for processing using the mvzip/mvexpand command. Like this
base search | streamstats count AS SrNo | eval z=mvzip(LogName, EventInLog, "^") | mvexpand z | rex field=z "(?<LogName>[^\^]+)\^(?<EventInLog>.*)" | fields - z | stats count by SrNo
I have create a Regex to Extract Hostname and replace the IP.
My Regex is ---- ^[^ \n]* ([^ ]+)
I would ingest the entire file as a single event and define appropriate mv field extractions. LIke this
props.conf
[unique_stanza]
REPORT-extract_fields = extract_mv_fields
transforms.conf
[extract_mv_fields]
REGEX=LogName\>\s([^\<]+)[\s\S]+?EventInLog\>\s([^\<]+)
FORMAT=$1::$2
MV_ADD=TRUE
Once you have the data in, in your SPL you can break the them apart into separate events for processing using the mvzip/mvexpand command. Like this
base search | streamstats count AS SrNo | eval z=mvzip(LogName, EventInLog, "^") | mvexpand z | rex field=z "(?<LogName>[^\^]+)\^(?<EventInLog>.*)" | fields - z | stats count by SrNo
Hi,
This might work indeed, I'll try this ASAP!
I completely forgot about mv field extractions and mvexpand...
The road to the knowledge is long ahah 🙂
Thanks for your help,
Cheers