How do I extract these fields from my data using rex?

[IRHM73]

Hi,

I wonder whether someone could help me please.

I have a field called detail.cid-repsonse which looks like the following:

[{"name":{"current":{"firstName":"JOHN","lastName":"SMITH"}},"ids":{"sut":"1234567890","nino":"AA111111A"},"dateOfBirth":"26121973"}]

From this I need to create new fields and extract the following data:

First Name
Last Name
Sut
NINO
DOB

I just wondered whether someone may be able to offer some guidance on how I may go about this please.

Any help would be greatly appreciated.

Many thanks and kind regards

Chris

[dmaislin_splunk]

Just use this technique:

http://docs.splunk.com/Documentation/Splunk/6.1.2/Data/Extractfieldsfromfileheadersatindextime

Forwarder props.conf entry for the specific sourcetype.
INDEXED_EXTRACTIONS=JSON

Fields are fed to indexers from the forwarder and searches will be much faster as a result.

[tom_frotscher]

I think he does not have json only. Just the field he mentioned in his questions is json. If the complete event is json, your anser might be the even better option.

[dmaislin_splunk]

Yes, it speeds up search and offloads the indexers from having to perform line-breaking and timestamp recognition tasks too.

[IRHM73]

Hi @dmaislin, thank you for taking the time to reply to my post. I'm very new to Splunk, so your solution may be a little over my head, but I really appreciate you highlighting something which I will no doubt be able to use in the future.

Many thanks and kind regards

Chris

[dmaislin_splunk]

No problem. If the logged events are JSON, this technique is probably the simplest approach as all of your fields will be present without requiring any extra field extraction work.

[tom_frotscher · Accepted solution]

Hi,

you simply can do this with the rex command. You can restrict the rex command to one field with the field parameter. Here an example for the first name:

... | rex field="detail.cid-repsonse" "\"firstName\":\"(?<firstName>[^\"]+)" | ...

I created a user everywhere example, which means, you can copy the follwoing search and paste it to your splunk search line and it will work. This can give you an idea how things work:

| stats count | eval "detail.cid-repsonse"="[{\"name\":{\"current\":{\"firstName\":\"JOHN\",\"lastName\":\"SMITH\"}},\"ids\":{\"sut\":\"1234567890\",\"nino\":\"AA111111A\"},\"dateOfBirth\":\"26121973\"}]" | rex field="detail.cid-repsonse" "\"firstName\":\"(?<firstName>[^\"]+)" | rex field="detail.cid-repsonse" "\"lastName\":\"(?<lastName>[^\"]+)" | rex field="detail.cid-repsonse" "\"sut\":\"(?<sut>[^\"]+)" | table "detail.cid-repsonse" firstName lastName sut

By the way, it looks like you have valid json in your field, so you might also be able to use the spath command: http://docs.splunk.com/Documentation/Splunk/6.2.5/SearchReference/Spath

In your case just append this to your search:

| spath input="detail.cid-repsonse"

and you will get new fields with your needed data.

Greetings

Tom

[IRHM73]

Hi Tom, this is great and works a treat.

Thank you for taking the time to reply to my post.

Kind Regards and thanks

Chris