How to configure Splunk 6.4.2 to extract this field within double quotes from my sample data as one field, not six separate fields?

[mspoerr]

Hello,

I have Message-Tracking Logs from Exchange 2016 servers where the fields are comma separated, but in some lines Microsoft uses Sub-Fields. In this case the Main field is quoted and a commas are used again as separators inside this main field.

Example:

2016-12-20T14:33:54.693Z,fe80::b9c4:56fa:d460:81f3,exchangesrv.test.com,fe80::b9c4:56fa:d460:81f3%12,exchangesrv,"MDB:bdfe3301-0f8f-48ce-92e4-6ff938ad1a6d, Mailbox:6a2dac65-cadc-46ec-b44c-98fba096c55e, Event:156022, MessageClass:IPM.Note.ProbeMessage.MBTSubmissionServiceHeartbeatProbe, CreationTime:2016-12-20T14:33:54.599Z, ClientType:Monitoring, ServerMdbConnectionId:08D423660D5DE7FD",,STOREDRIVER,RECEIVE,2014,,4c7b2b2e-291a-4635-f603-08d428e539b5,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,To,8944,1,,,0000003a-0000-0000-0000-0000f519e953-MBTSubmissionServiceHeartbeatProbe,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,04I: ,Originating,,,,S:MailboxDatabaseGuid=bdfe3301-0f8f-48ce-92e4-6ff938ad1a6d;S:ItemEntryId=00-00-00-00-2D-E8-87-0C-DF-D8-A0-42-97-64-3D-D7-57-8C-2A-3C-07-00-E7-FB-62-04-97-51-5F-44-B6-C5-9B-A3-65-97-9D-5F-00-00-00-00-01-0B-00-00-E7-FB-62-04-97-51-5F-44-B6-C5-9B-A3-65-97-9D-5F-00-00-1E-84-2C-F6-00-00;S:DeliveryPriority=Normal;S:AccountForest=test.com;S:IsProbe=true;S:PersistProbeTrace=False,Email,940aee54-2531-41e4-f603-08d428e539b5,15.01.0544.027

This part below should be just one field:

"MDB:bdfe3301-0f8f-48ce-92e4-6ff938ad1a6d, Mailbox:6a2dac65-cadc-46ec-b44c-98fba096c55e, Event:156022, MessageClass:IPM.Note.ProbeMessage.MBTSubmissionServiceHeartbeatProbe, CreationTime:2016-12-20T14:33:54.599Z, ClientType:Monitoring, ServerMdbConnectionId:08D423660D5DE7FD"

The problem now is that Splunk (6.4.2) does not handle the quoted field as one field - it splits it up to 6 separate fields. Can I change that behavior? I found the following question where Splunk does the magic I want and interprets everything within the quotes as one field:
https://answers.splunk.com/answers/99398/delims-fields-with-a-field-that-has-sub-fields.html?utm_sou...

Is there an option in transforms.conf I am missing?

Thanks,
/mspoerr

[mspoerr]

In a second try it suddenly works. I am not sure if I just was impatient or I have overseen something when I configured it the first time...

[nabeel652]

A quick workaround should be to extract the quoted field separately from the _raw data. You can do it through field extractions or props.conf and transforms.conf. Here I would give you the example using inline rex command:

-- Your query -- | eval _raw = " 2016-12-20T14:33:54.693Z,fe80::b9c4:56fa:d460:81f3,exchangesrv.test.com,fe80::b9c4:56fa:d460:81f3%12,exchangesrv,\"MDB:bdfe3301-0f8f-48ce-92e4-6ff938ad1a6d, Mailbox:6a2dac65-cadc-46ec-b44c-98fba096c55e, Event:156022, MessageClass:IPM.Note.ProbeMessage.MBTSubmissionServiceHeartbeatProbe, CreationTime:2016-12-20T14:33:54.599Z, ClientType:Monitoring, ServerMdbConnectionId:08D423660D5DE7FD\",,STOREDRIVER,RECEIVE,2014,,4c7b2b2e-291a-4635-f603-08d428e539b5,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,To,8944,1,,,0000003a-0000-0000-0000-0000f519e953-MBTSubmissionServiceHeartbeatProbe,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,HealthMailbox1b30cca7073e4d0c8d2b96a01198b492@test.com,04I: ,Originating,,,,S:MailboxDatabaseGuid=bdfe3301-0f8f-48ce-92e4-6ff938ad1a6d;S:ItemEntryId=00-00-00-00-2D-E8-87-0C-DF-D8-A0-42-97-64-3D-D7-57-8C-2A-3C-07-00-E7-FB-62-04-97-51-5F-44-B6-C5-9B-A3-65-97-9D-5F-00-00-00-00-01-0B-00-00-E7-FB-62-04-97-51-5F-44-B6-C5-9B-A3-65-97-9D-5F-00-00-1E-84-2C-F6-00-00;S:DeliveryPriority=Normal;S:AccountForest=test.com;S:IsProbe=true;S:PersistProbeTrace=False,Email,940aee54-2531-41e4-f603-08d428e539b5,15.01.0544.027" | rex field=_raw "\"(?<quoted>.*)\"" | table _raw quoted

[mspoerr]

Thanks for your comment but it seems I wasn't specific enough. I would like to extract all 30 fields not just the special one.

[nabeel652]

There may be a better solution but as a workaround I think use automatically extracted fields for all other fields and extract the field in quotes separately from raw data.