Splunk Search

How to extract dynamic key value pairs in mixed data from DB Connect app?

TangentTexan
New Member

I am indexing a string for the DBConnect where one of the fields stores a modified data in one of the cells.
In a sub part of a Log entry I am having a difficulty extracting the KV pairs. Where the previous parts of each log entry have the basic Key=Value on a line each, that is working. The part that is not is from the AllXML field - string= item.

  1. Starts with:
    string="
  2. Each key:value pair is separated with:
    :
  3. Additional Keys begin from:
    

  4. Additional Values end with:
    
  5. Stops with:
    "

Sample Log Entry:

AllXML=<error
    application="my application name"
    detail="whole bunch of stuff"
    time="2014-08-08T11:11:59.4225842Z"
    statusCode="500">   
<serverVariables>
        <item
          name="ALL_HTTP">
          <value    
    string="KEYNAME1:Value1 Value1,MoreValue1&#xD;&#xA;Key-Name2:Value2-Value2.Value2&#xD;&#xA;KEY-NAME3:true&#xD;&#xA;Key_Name4:Value4 ; Value4,Value4.Value4 - vvvvvvAAALLLLuuuueeeee44444&#xD;&#xA;" />

The OutPut should be, so that these items get indexed:

application="my application name"
detail="whole bunch of stuff
time=2014-08-08T11:11:59.4225842Z
statusCode=500
KEYNAME1=Value1
Key-Name2=Value2-Value2.Value2
KEY-NAME3=true
Key_Name4=Value4 ; Value4,Value4.Value4 - vvvvvvAAALLLLuuuueeeee44444

I know it should be through the transform.conf - but getting the regex to pull the data in has become difficult.
ant assistance would be appreciated.
Thanks,

0 Karma

somesoni2
Revered Legend

Try this

In props.conf

[YourSourceType]
REPORT-customkvtransform = customkvextract

In transforms.conf

[customkvextract]
CLEAN_KEYS = 1
FORMAT = $2::$3
MV_ADD = 0
REGEX = ([&#xA;]*)([^:]+):([^&]+)&#xD;
SOURCE_KEY = string
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...