Splunk Search

How to get multiple dynamic values from a single log line

littlgra
Engager

We have numerous log lines that are in a format similar to the following:-

2019-04-09 13:00:03 DEBUG DynamicName1 1000 (1.00) ; DynamicName2 2000 (2.00) ; 
2019-04-09 13:00:02 DEBUG DynamicName2 500 (0.50) ; DynamicName4 3100 (3.10) ; DynamicName5 12000 (12.00) ;
2019-04-09 13:00:00 DEBUG DynamicName1 600 (0.60) ; DynamicName5 2100 (2.10) ;

The DynamicName# is a dynamic string that can have multiple values per line (but never the same value per line), the numbers after it represent a timing in milliseconds and then seconds.

What I want to get is a table of all the unique DynamicName(s), their average execution times and counts

However, I can't quite get the extraction correct. When I use a rex, for example

rex field=_raw "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"  | table name time

However this creates a table of multiple values per row and then I can't use other commands on it correctly. For example:-

rex field=_raw "(?<name>\w+) (?<time>\d+) \(\d+.\d+\) ; "  | table name time | sort -time

Does not result in the correct result I am expecting.

Is there a way I can correctly extract the data to get true dynamic multiple values that I can then table with 1 DynamicName per table row

1 Solution

grittonc
Contributor

Try splitting it up into a mv field after stripping out the first characters that aren't needed:

| eval foo=replace(_raw, "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} DEBUG", "")
| makemv delim=";" foo
| mvexpand foo
| rex field=foo "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"
| table name time | sort -time

Then you can use mvexpand to split it up into multiple events and your regex can work on that.

View solution in original post

0 Karma

grittonc
Contributor

Try splitting it up into a mv field after stripping out the first characters that aren't needed:

| eval foo=replace(_raw, "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} DEBUG", "")
| makemv delim=";" foo
| mvexpand foo
| rex field=foo "(?<name>\w+) (?<time>\d+) \(\d+.\d+\)"
| table name time | sort -time

Then you can use mvexpand to split it up into multiple events and your regex can work on that.

0 Karma

littlgra
Engager

Awesome that worked. I had played with the mv functions before but couldn't get it to work. Much appreciated

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...