Using KV_MODE=auto in props.conf, how do I get a search-time extraction to work for key-value pairs with a preceding "?" character?

[skovalenko]

I use kv_mode = auto in my props.conf and it works most of the time. The only time when it fails to extract is when there is a ? in front of the key=value. Here is the example

url = http://blahblah.net/live/md-global/external/1p_regprofile/livenation1p_regprofile_3128.ts?a=value1&b...

The fields are extracted like this:
url http://blahblah.net/live/md-global/external/1p_regprofile/livenation1p_regprofile_3128.ts?a=value1
b value2
c value3

Is there any trick to get a=value1 to get extracted as well?

[gcato]

Hi skovalenko,

You can configure an entry in transforms.conf and specify multiple delimiters for the key=value (kv) pairs in the event. Here's the example entry from the docs http://docs.splunk.com/Documentation/Splunk/6.3.0/Admin/Transformsconf

[multiple_delims]
DELIMS = "|;", "=:"

The above example extracts key-value pairs which are separated by '|' or ';'.

while the key is delimited from value by '=' or ':'.

You can test the DELIMS configuration you need with the kv command, like so...

|stats count as _raw | eval _raw = "url = http://blahblah.net/live/md-global/external/1p_regprofile/livenation1p_regprofile_3128.ts?a=value1&b...; | kv pairdelim="?&" kvdelim="=" |table *

To automate, you will need to add a REPORT entry in props.conf to point to the transforms.conf entry and also restart Splunk (or try "http://you_splunk_server:8000/en_GB/debug/refresh" first to reload the configuration).

Here's a good blog entry on extracting kv entries - http://blogs.splunk.com/2008/02/12/delimiter-based-key-value-pair-extraction/

[skovalenko]

Thank you gcato,

It worked for the extraction part, now I see the a=value1 as a field. However, the url field still remained url = http://blahblah.net/live/md-global/external/1p_regprofile/livenation1p_regprofile_3128.ts?a=value1

But, if I test using kv command, the url fields before "?" url = http://blahblah.net/live/md-global/external/1p_regprofile/livenation1p_regprofile_3128.ts