Splunk Search

Create key value pairs from existing fields

ben_leung
Builder

In _raw:

string1=key1|key2|key3|key4|key5|key6 string2=value1|value2|value3|value4|value5|value6

I want to manipulate/rearrange and get an output that is like this.

key1=value1
key2=value2
key3=value3
...

Tags (3)

somesoni2
SplunkTrust
SplunkTrust

If this is your _raw data

2014-10-23 22:25:41 string1=key1|key2|key3  string2=value1|value2|value3
 2014-10-23 22:26:00 string1=key4|key5  string2=value4|value5

then try this (runanywhere sample, till eval _time=now() is just for generating dummy data, this will be your base search)

| gentimes start=-1 | eval temp="2014-10-23 22:25:41 string1=key1|key2|key3  string2=value1|value2|value3#2014-10-23 22:26:00 string1=key4|key5  string2=value4|value5" | table temp | makemv temp delim="#" | mvexpand temp | rename temp as _raw| extract pairdelim=" ", kvdelim="=:" | eval _time=now()
| eval string1="timestamp|".string1 | eval string2=_time."|".string2 | makemv delim="|" string1 | makemv delim="|" string2 | eval temp=mvzip(string1,string2,"=") | nomv temp | rename temp as _raw | table _raw| extract pairdelim=" ", kvdelim="=:"
0 Karma

jkat54
SplunkTrust
SplunkTrust

Hi, this should get you in the right direction:

...|rex field=_raw "string1=(?<field1>\w+\d+)\|(?<field3>\w+\d+)\|(?<field5>\w+\d+)\|(?<field7>\w+\d+)\|(?<field9>\w+\d+)\|(?<field11>\w+\d+)\sstring2=(?<field2>\w+\d+)\|(?<field4>\w+\d+)\|(?<field6>\w+\d+)\|(?<field8>\w+\d+)\|(?<field10>\w+\d+)\|(?<field12>\w+\d+)$" | eval _raw=field1."=".field2.",".field3."=".field4.",".field5."=".field6.",".field7."=".field8.",".field9."=".field10.",".field11."=".field12
0 Karma

ben_leung
Builder

The challenge is that for string1 and string 2, there are random number of values, although they will have the same number of them.

There are 3 in the first event and then 2 in the next.

2014-10-23 22:25:41 string1=key1|key2|key3  string2=value1|value2|value3
2014-10-23 22:26:00 string1=key4|key5  string2=value4|value5
0 Karma

jkat54
SplunkTrust
SplunkTrust

Dude... what is up with this data!

This is probably easier another way, but this example supports up to 10 key values and rewrites the timestamp as well, note i separate the date from the other strings with a colon in the eval, you might want to change it to a space. Finally we remove the extra ,= if they exist using sed:

...|rex field=_raw "(^.*(?=string1=))string1=(?<f1>\w+)?\|?(?<f2>\w+)?\|?(?<f3>\w+)?\|?(?<f4>\w+)?\|?(?<f5>\w+)?\|?(?<f6>\w+)?\|?(?<f7>\w+)?\|?(?<f8>\w+)?\|?(?<f9>\w+)?\|?(?<f10>\w+)?\|?\s+string2=(?<f11>\w+)?\|?(?<f12>\w+)?\|?(?<f13>\w+)?\|?(?<f14>\w+)?\|?(?<f15>\w+)?\|?(?<f16>\w+)?\|?(?<f17>\w+)?\|?(?<f18>\w+)?\|?(?<f19>\w+)?\|?(?<f20>\w+)?\|?$" | eval _raw=f1.":".f2.=".f12.",".f3."=".f13.",".f4."=".f14.",".f5."=".f15.",".f6."=".f16.",".f7."=".f17.",".f8."=".f18.",".f9."=".f19.",".f10."=".f20.",".f11."=".f21 | rex mode=sed field=_raw "s/(\,=)+//g"

Works in theory!

0 Karma

ben_leung
Builder

The office politics with developers..

0 Karma

jkat54
SplunkTrust
SplunkTrust

does the crazy expensive search above work for now? Are you ok with the commas between the key value pairs or did you absolutely need line breaks?

0 Karma

Raghav2384
Motivator

May be this...

base search| makemv delim="|" System1|makemv delim="|" System2|eval fields = mvzip(System1,System2)|mvexpand fields|rex field=fields "(?\w+|\d+),(?\w+|\d+)"|eval New_Field = alpha + "=" + beta|table alpha,beta,New_Field

0 Karma

ben_leung
Builder
base search| makemv delim="|" System1|makemv delim="|" System2|eval fields = mvzip(System1,System2)|mvexpand fields

The new "fields" field has what I need but only gets the first fields="key1=vlaue1". The other fields and values are not grabbed since the mvzip argument in the eval command only grabs the first.

Not sure what is the purpose for the rex command and everything after that.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...