Splunk Search

Regex for items in single quotes

aramakrishnan
New Member

I have a bunch of logs which look like the following.

2017-05-01 18:36:16,885 UTC - DEBUG - testpod_service.segment_processor - [BTESTC4277:37223] returning requests="[{'status': 'request_automat_eventab', 'priority': -2868.565, 'confidence': 1.0, 'db': {'dur_sec': 2868.565, 'n_hours': 2}, 'onset': 66377515, 'event_name': 'CYCLE', 'offset': 66951227

For every segment that looks like 'field':'value'I want Splunk to treat the value in quotes BEFORE the : as the field name and the value in quotes AFTER the : as the value for the field (similar treatment to field="value", which is Splunk's default).

How do I achieve this through a regex treatment? For example, in the example above, I'd like my field and value pairs to be as follows:

status = request_automat_eventab
priority = -2868.565
confidence = 1.0
dur_sec=2868.565
n_hours=2

...and so on

Thanks in advance!

0 Karma

kmorris_splunk
Splunk Employee
Splunk Employee

You can use the _KEY_1 and _VAL_1 field names in your regex capture groups to get all of the key value pairs dynamically. For example, in your props.conf, under the sourcetype for these events:

[yoursourcetype]
EXTRACT-keysandvalues = \"\[{\'(?<_KEY_1>\S+)\': \'(?<_VAL_1>\S+)\'

This will take every value in single quotes before the colon as the field name and the value in single quotes after the colon as the value for that field. This will work for any format you can match in the extractions regex. As long as the key value pairs are consistent, it will grab them all.

Hope this helps.

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Are you asking about at index time or at search time?

0 Karma

kmorris_splunk
Splunk Employee
Splunk Employee

If the field / value format is consistent, you can use <_KEY_1> and <_VAL_1> as your field names in your capture groups to dynamically extract all field value pairs. In your props.conf, under the sourcetype for these events add a field extraction like the following:

[yoursourcetypehere]
EXTRACT-keysandvals = \"\[{\'(?<_KEY_1>\S+)\': \'(?<_VAL_1>\S+)\'

You may have to play with the regex a bit, but this should work. It will take everything in single quotes before the colon as a field name and everything after the colon in single quotes as the value for that field. You can use this any time you have a consistent format for the pairs.

I had a customer example where the field was in brackets followed by the value:

[fieldname1]fieldvalue1[fieldname2]fieldvalue2

Hope this helps.

0 Karma

adonio
Ultra Champion

use the field extractor from GUIi? click an event -> actions -> extract fields -> new tab pops -> follow the prompts -> save -> search your fields
read more here:
http://docs.splunk.com/Documentation/Splunk/6.5.3/Knowledge/ExtractfieldsinteractivelywithIFX

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...