Splunk labels in table format

[vadlamudi]

Hi There,

Log event:

[ 2021-02-04 23:14:28.925 SingleApp log:158] 200 GET /apache/proxy/user/1123123/qsdddqwedqewdqwdqwd/api (@1.1.1.1.1) 0.98ms
k8s_label = app=jupyterhub k8s_label = component=server k8s_label = entity_type=jupyter-lab k8s_label = heritage=jupyterhub k8s_label = identity_name=namespace k8s_label = notebook_id=1123324234 k8s_label = notebook_name=learningpython.iypnb k8s_label = project_id=123 k8s_label = sdr.appname=notebooks k8s_label = user=mattparker k8s_label = user_id=1123234 k8s_label = zone=us-east-2b

I am trying to achieve the k8s_labels in table format in splunk but however, when i us the below query i am getting an error.  any help would be appreciated

Query:

index=<index_name> sourcetype=<sourcetype_Name> 1123123 | table k8s_label="user_id=1123234",k8s_label="mattparker"

Error:

Error in 'table' command: Invalid argument: 'k8s_label=user_id=1123234'

Thank You

[vadlamudi]

Thank you for the query @somesoni2 . When i run the query, i am getting the below output.  It is not extracting the K8s_labels.

[ 2021-02-04 23:14:28.925 SingleApp log:158] 200 GET /apache/proxy/user/1123123/qsdddqwedqewdqwdqwd/api (@1.1.1.1.1) 0.98ms

[vadlamudi]

@somesoni2   Below is the expected table format from the key value pairs.

Log event:

[ 2021-02-04 23:14:28.925 SingleApp log:158] 200 GET /apache/proxy/user/1123123/qsdddqwedqewdqwdqwd/api (@1.1.1.1.1) 0.98ms
k8s_label = app=jupyterhub k8s_label = component=server k8s_label = entity_type=jupyter-lab k8s_label = heritage=jupyterhub k8s_label = identity_name=namespace k8s_label = notebook_id=1123324234 k8s_label = notebook_name=learningpython.iypnb k8s_label = project_id=123 k8s_label = sdr.appname=notebooks k8s_label = user=mattparker k8s_label = user_id=1123234 k8s_label = zone=us-east-2b

Expected Ouptut:

The k8s_labels from the log event should be in table format.  below is the sample output of k8s_label.

notebook_name       earningpython.iypnb

project_id                     123

user                                   mattparker

[somesoni2]

Give this a try

index=yourIndex sourcetype=yourSourcetype ..other search criteria
| table _raw
| rex field=_raw mode=sed "s/k8s_label\s*\=\s*//g" | extract

You're basically getting rid of repeating phrase "k8s_label = " as it's interfering with automatic field extraction. The output will contains field names like app, component, entitiy_type  etc, with corresponding values. See if that works for you.

[somesoni2]

Can you provide a sample output (the table that you want your Splunk search to return) based on your sample data? A search time transformation of the data could be possible but would need to know the exact output to attempt any solution.

[vadlamudi]

i will be unable to add the transforms (no access to the splunk infra).  Therefore, i am looking for the search time commands. Any direction to start will be helpful.

[PickleRick]

You can add the extractions from the GUI (the settings you add there land in the props.conf and transforms.conf of the search app). But in search time...

Try this:

| rex max_match=0 "k8s_label\s*=\s*(?<myprefix>[^=]+)=(?<mysuffix>\S+)" 
| eval myvars=mvzip(myprefix,mysuffix)
| mvexpand myvars 
| rex field=myvars "(?<mykey>[^,]+),(?<myvalue>.*)"
| eval {mykey}=myvalue 
| fields - mykey myvalue myprefix mysuffix myvars
| stats values(*) by _raw _time 
| foreach values(*)
    [ rename <<FIELD>> as <<MATCHSTR>> ]

Ugly like hell but might just work 😉

[vadlamudi]

Hi @PickleRick  sorry for the confusion.  What i trying to achieve here is that something like below in tabular format. i am trying to put the key value pairs in tabular format.

notebook_name   learningpython.iypnb

project_id            123

user                          mattparker

[PickleRick]

OK. So in other words you want to parse out key/value pairs from each "entity" given as value to k8s_label, right?

It would be easiest if you added a proper entry into props.conf and transforms.conf to create fields dynamicaly.

Otherwise, just running search-time commands, it will be complicated. Probably possible, but complicated.

Unless the set of field and their order is known and more-or-less constant.

[PickleRick]

To be honest - I have no idea what you're trying to do.

Firstly, regardless of what you want, your search doesn't make much sense. You add conditions in the search, you table with the table, but you don't filter with the table. So if you want to restrict your results (return only a subset of maching events), you add the condition to the search in front of the pipe character. To the table command you only give names of fields which are to be put into the table.

But there's more - I don't understand what you're trying to do with your data. You have several chunks of data there with the same "name" and different values which are key/value pairs. Are they being parsed into fields? Or maybe you want to parse them? If so, you have to let splunk know how to do so (and table command is definitely not the way to go).