Splunk Search

Why are delimited field extractions not working?

richardAtOmni
Path Finder

Hello, we are inputting data via the HTTP Event collector. The "event" member has this format, which we are trying to split into fields with the pipe delimiter:

{"data":"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\\SYSTEM|No sync required between CA and XA||Logger"}

As I work through the field extraction definition tool, the delimiter properly splits out the fields. I work through and I rename each field the way I want. Then, I save the field extraction. I get a message saying that this was successful. Then I click on the link "explore the fields that I just extracted" (I'm paraphrasing from memory), then it takes me to a search with a filter on the sourcetype that I just defined the field extraction for.

The problem is the search results do not show the new fields that I just defined. It only shows the first one. And as it's value it has the entire row as the value, as though none of the delimiters were recognized at all.

For example, on the row above, if my first field was named "start", then it would have a value of:

{"data":"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\\SYSTEM|No sync required between CA and XA||Logger"}

What am I missing?

Thanks for any insight you can provide.

0 Karma
1 Solution

richardAtOmni
Path Finder

The delimited fields worked as expected after we changes our input format to be just a straight string, instead of a nested JSON object.

So instead of this:

{"data":"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\SYSTEM|No sync required between CA and XA||Logger"}

We have this:
"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\SYSTEM|No sync required between CA and XA||Logger"

I'm not sure, but there seems to have been something about the nested JSON which prevented the parsing from working as expected?

The original question of why this didn't work isn't quite answered, but we're good for our use cases to proceed, so I'll mark it closed.

Thanks!

View solution in original post

0 Karma

richardAtOmni
Path Finder

The delimited fields worked as expected after we changes our input format to be just a straight string, instead of a nested JSON object.

So instead of this:

{"data":"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\SYSTEM|No sync required between CA and XA||Logger"}

We have this:
"Omni.Riva.CrmAgentEx.exe (ci15)|NFRwlv #6218|3940170112032121|RC-SRV3||40000|14A21XFJS85PQ|NT AUTHORITY\SYSTEM|No sync required between CA and XA||Logger"

I'm not sure, but there seems to have been something about the nested JSON which prevented the parsing from working as expected?

The original question of why this didn't work isn't quite answered, but we're good for our use cases to proceed, so I'll mark it closed.

Thanks!

0 Karma

mpreddy
Communicator

Hi,

Use props and transforms in search time it will extract | seperated.
props.conf

[sourcetype]
REPORT-fields = pipefields

transforms.conf

[pipefields]
DELIMS = "|"
FIELDS = field1, field2, field3, field4, field5, field6, field7, field8, field9

jawaharas
Motivator

Your suggestion helped me. Thank you. 🙂

0 Karma

richardAtOmni
Path Finder

I checked these files, and the field extractor tool I used to define the delimited fields pretty much generated the same config that you suggest. However, it still doesn't work for some reason.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...