Splunk Search

Extract using pairdelim and kvdelim

dbcase
Motivator

Hi, First time trying to use pardelim and kvdelim and having no luck.

The data looks like this

####<Aug 19, 2016 3:06:01 PM CDT> <Debug> <ucontrol> <betamax-portal1> <managedServer3> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1471637161846> <BEA-000000> <com.ucontrol.rest.security.LoggingFilter  - RCVD REST request: 
GET /rest/icontrol/login HTTP/1.1
Host: betamax-portal1:8080
accept-language: en-us
user-agent: Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13C75 (2037495216)
x-login: cccc
x-password: ********
accept: application/json, text/plain, */*
x-allownonactivatedlogin: true
referer: http://10.0.12.143:8100/
x-appkey: defaultKey
x-expires: 600000
cookie: _ga=GA1.1.1016043735.1471636804; _gat=1
accept-encoding: gzip, deflate
Via: 1.1 beta.icontrol.com
X-Forwarded-For: 63.239.122.42
X-Forwarded-Host: beta.icontrol.com
X-Forwarded-Server: beta.icontrol.com
Connection: close
expand: sites,instances,points,functions

And the nth iteration of my non working query looks like this. The query executes but I don't see any of the fields.

index=cox sourcetype=wls_managedserver Stuck| kv pairdelim="\n" kvdelim=":"
0 Karma
1 Solution

sundareshr
Legend

Try this

index=cox sourcetype=wls_managedserver Stuck| extract pairdelim="\n" kvdelim=":"

View solution in original post

0 Karma

sundareshr
Legend

Try this

index=cox sourcetype=wls_managedserver Stuck| extract pairdelim="\n" kvdelim=":"
0 Karma

dbcase
Motivator

Hi Sundareshr!

Do you ever sleep? 🙂 The query executes but I don't see any of the fields in the field list to the left. I really haven't used this before so I'm pretty sure I'm doing something wrong.... 🙂

0 Karma

dbcase
Motivator

I'm trying to pull the values for Content-Type but every time I try to use that field it comes up null, even though the data clearly has values

0 Karma

sundareshr
Legend

🙂 I don't see Content-Type in the sample you provided? What am I not looking at?

One of the things you may already be aware of, the fields on the left, only show if more than 20% events have the field.

0 Karma

dbcase
Motivator

Hi Sundareshr,

I think I pasted the wrong one before. Below is the correct one. The search only returns 2 events and both of them have the Content-Type kv.

####<Aug 11, 2016 6:45:27 AM EDT> <Error> <WebLogicServer> <ccivirpxa0712> <managedServer23> <[ACTIVE] ExecuteThread: '20' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1470912327355> <BEA-000337> <[STUCK] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "662" seconds working on the request "Workmanager: mediaUpload-workManager, Version: 0, Scheduled=true, Started=true, Started time: 662872 ms
[
POST /cameraProxy/video?token=MTE0NTg5OjExNDU4OS4xNDcwMzgzNzc1OjExNDU4OS4zOjExNDU4OV83ODAzNDM2NTE3X3ZfMTM2NF80MjM4MjUwOTo2MDAyNTM= HTTP/1.1
Content-Type: video/mp4
Authorization: Basic OTQ0YTBjMjUzMDgxOg==
X-TimestampedFile: 160811023334
X-Capture-Time: 160811023334
Date: Thu, 11 Aug 2016 02:33:39 -0800
X-TriggerType: http
X-EventInfo: httpc_1
Via: 1.0 10.210.192.63
X-Forwarded-For: 10.210.192.5
X-Forwarded-Host: 184.185.0.36
X-Forwarded-Server: 10.210.192.63
Connection: Keep-Alive
Content-Length: 1542184
]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
null> 
Collapse
0 Karma

dbcase
Motivator

so a follow on question. If the field doesn't appear in 20% of the total events (this would be true in this case as there are 1000's of events but only 2 have the Stuck keyword), how can the field be extracted and used?

0 Karma

dbcase
Motivator

Oh Goody! TWO things I've not done before 🙂

Ok I've added the field transformation and re-run the search but still no luck. Is there something else I should be doing?

0 Karma

sundareshr
Legend

Did you restart splunk on SH?

0 Karma

sundareshr
Legend

If it is only the Content-Type field you're interested in, may be easier to just use rex

... | rex "Content-Type:\s+(?<contenttype>[^\n\r]+)"
0 Karma

sundareshr
Legend

Not sure why that is not working. You can achieve the same result by adding a field transformation (http://docs.splunk.com/Documentation/Splunk/6.4.2/Knowledge/Managefieldtransforms)

Regular Expression: ([\w+-[^:]+):\s+([^\n]+)
Format: $1::$2
0 Karma

dbcase
Motivator

Actually the data looks like this....#### <[ACTIVE] ExecuteThread: '20' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1470912327355> <[STUCK] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "662" seconds working on the request "Workmanager: mediaUpload-workManager, Version: 0, Scheduled=true, Started=true, Started time: 662872 ms
[
POST /cameraProxy/video?token=MTE0NTg5OjExNDU4OS4xNDcwMzgzNzc1OjExNDU4OS4zOjExNDU4OV83ODAzNDM2NTE3X3ZfMTM2NF80MjM4MjUwOTo2MDAyNTM= HTTP/1.1
Content-Type: video/mp4
Authorization: Basic OTQ0YTBjMjUzMDgxOg==
X-TimestampedFile: 160811023334
X-Capture-Time: 160811023334
Date: Thu, 11 Aug 2016 02:33:39 -0800
X-TriggerType: http
X-EventInfo: httpc_1
Via: 1.0 10.210.192.63
X-Forwarded-For: 10.210.192.5
X-Forwarded-Host: 184.185.0.36
X-Forwarded-Server: 10.210.192.63
Connection: Keep-Alive
Content-Length: 1542184
]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
null>

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...