Splunk Search

How to get multiple dynamic values from a single log line

littlgra
Engager

We have numerous log lines that are in a format similar to the following:-

2019-04-09 13:00:03 DEBUG DynamicName1 1000 (1.00) ; DynamicName2 2000 (2.00) ; 
2019-04-09 13:00:02 DEBUG DynamicName2 500 (0.50) ; DynamicName4 3100 (3.10) ; DynamicName5 12000 (12.00) ;
2019-04-09 13:00:00 DEBUG DynamicName1 600 (0.60) ; DynamicName5 2100 (2.10) ;

The DynamicName# is a dynamic string that can have multiple values per line (but never the same value per line), the numbers after it represent a timing in milliseconds and then seconds.

What I want to get is a table of all the unique DynamicName(s), their average execution times and counts

However, I can't quite get the extraction correct. When I use a rex, for example

rex field=_raw "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"  | table name time

However this creates a table of multiple values per row and then I can't use other commands on it correctly. For example:-

rex field=_raw "(?<name>\w+) (?<time>\d+) \(\d+.\d+\) ; "  | table name time | sort -time

Does not result in the correct result I am expecting.

Is there a way I can correctly extract the data to get true dynamic multiple values that I can then table with 1 DynamicName per table row

1 Solution

grittonc
Contributor

Try splitting it up into a mv field after stripping out the first characters that aren't needed:

| eval foo=replace(_raw, "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} DEBUG", "")
| makemv delim=";" foo
| mvexpand foo
| rex field=foo "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"
| table name time | sort -time

Then you can use mvexpand to split it up into multiple events and your regex can work on that.

View solution in original post

0 Karma

grittonc
Contributor

Try splitting it up into a mv field after stripping out the first characters that aren't needed:

| eval foo=replace(_raw, "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} DEBUG", "")
| makemv delim=";" foo
| mvexpand foo
| rex field=foo "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"
| table name time | sort -time

Then you can use mvexpand to split it up into multiple events and your regex can work on that.

0 Karma

littlgra
Engager

Awesome that worked. I had played with the mv functions before but couldn't get it to work. Much appreciated

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...