Splunk Search

Extracting multiple similar values from a multi-line event

mikelanghorst
Motivator

I've got a rather tricky (at least for me) data set that I'd like to extract values from. For this example text

`

Elapsed Time Unti Event Locked Display :: Longest 5 Entries:
Elapsed Time Unti Event Locked Display :: [Max(0.361), Avg(0.180), Min(0.000)] sec(s) # Total: 00002 MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_RTD_RTD
Elapsed Time Unti Event Locked Display :: [Max(0.001), Avg(0.000), Min(0.000)] sec(s) # Total: 00007 MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_WFC_STATUS
Elapsed Time Unti Event Locked Display :: [Max(0.000), Avg(0.000), Min(0.000)] sec(s) # Total: 00001 MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_RTPD_RTPD
Elapsed Time Unti Event Locked Display :: [Max(0.000), Avg(0.000), Min(0.000)] sec(s) # Total: 00002 MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_RTD_REVIEW_PERIOD
Elapsed Time Unti Event Locked Display :: [Max(0.000), Avg(0.000), Min(0.000)] sec(s) # Total: 00002 MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_RTPD_REVIEW_PERIOD
>>> Eventing Processing Time :: Longest 5 Entries:
Eventing Processing Time :: [Max(0.421), Avg(0.308), Min(0.194)] sec(s) # Total: 00002 MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_RTD_REVIEW_PERIOD
Eventing Processing Time :: [Max(0.198), Avg(0.194), Min(0.190)] sec(s) # Total: 00002 MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_RTPD_REVIEW_PERIOD
Eventing Processing Time :: [Max(0.149), Avg(0.142), Min(0.134)] sec(s) # Total: 00002 MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_RTD_RTD
Eventing Processing Time :: [Max(0.039), Avg(0.039), Min(0.039)] sec(s) # Total: 00001 MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_RTPD_RTPD
Eventing Processing Time :: [Max(0.017), Avg(0.011), Min(0.000)] sec(s) # Total: 00007 MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_WFC_STATUS
User Sessions :: Number of sessions 1
`

I need to extract out the Max,Avg,Min values for each MenuId: in Eventing Processing Time. Max isn't a multivalue field, but rather only relevent to the MenuId in the same line. So I'd need something like RT_RTD_REVIEW_PERIOD-Max and RT_RTD_REVIEW_PERIOD-Avg.

I'm not sure how to do this other than simple brute forcing with multiple regexes.

Tags (1)
0 Karma
1 Solution

emiller42
Motivator

I believe the following would work, but only as an index-time extraction. (Not tested)

in Transforms.conf

[stanza_name]
REPEAT_MATCH = true
REGEX = Elapsed\sProcessing\sTime\s::\s\[\w+\(([^\)]+)\),\s\w+\(([^\)]+)\),\s\w+\(([^\)]+)\)\].+?#\s\w+:\s(\d+)\s+\w+:\s[\w\.]+-([\w_]+)
FORMAT = $5-Max::$1 $5-Avg::$2 $5-Min::$3 $5-Total::$4

My regex may not be the most clean/efficient, but it appears to capture everything correctly on regexr.

So from:

Eventing Processing Time :: [Max(0.421), Avg(0.308), Min(0.194)] sec(s)      # Total: 00002      MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_RTD_REVIEW_PERIOD

it will extract:

$1 = 0.421
$2 = 0.308
$3 = 0.194
$4 = 00002
$5 = RT_RTD_REVIEW_PERIOD

So $5-Max::$1 should become 'RT_RTD_REVIEW_PERIOD-Max = 0.421'

According to the documentation on transforms.conf, you can only do concatenated fields with index-time extractions.

View solution in original post

emiller42
Motivator

I believe the following would work, but only as an index-time extraction. (Not tested)

in Transforms.conf

[stanza_name]
REPEAT_MATCH = true
REGEX = Elapsed\sProcessing\sTime\s::\s\[\w+\(([^\)]+)\),\s\w+\(([^\)]+)\),\s\w+\(([^\)]+)\)\].+?#\s\w+:\s(\d+)\s+\w+:\s[\w\.]+-([\w_]+)
FORMAT = $5-Max::$1 $5-Avg::$2 $5-Min::$3 $5-Total::$4

My regex may not be the most clean/efficient, but it appears to capture everything correctly on regexr.

So from:

Eventing Processing Time :: [Max(0.421), Avg(0.308), Min(0.194)] sec(s)      # Total: 00002      MenuId: rtdMenu.rtnIntervalDispatch.rtnIntervalDispExecuctionControl-RT_RTD_REVIEW_PERIOD

it will extract:

$1 = 0.421
$2 = 0.308
$3 = 0.194
$4 = 00002
$5 = RT_RTD_REVIEW_PERIOD

So $5-Max::$1 should become 'RT_RTD_REVIEW_PERIOD-Max = 0.421'

According to the documentation on transforms.conf, you can only do concatenated fields with index-time extractions.

mikelanghorst
Motivator

Hmm, only pulling the first line. REPEAT_MATCH = true is set, but no affect.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...