Getting Data In

How to handle parsing of CSV files with fields containing commas?

bb2324oo
Engager

My data is parsed into CSV files with default comma delimiters.

Many fields contain one or more commas.

When I use this in props.conf:

    [your_sourcetype]
   EXTRACT-second_round = ^(?[^,]*),(?[^,]*),(?[^,]*),(?.*),(?[^,]*),(?[^,]*),(?[^,]*)

However, Commas could only be in field 4, it does not support multiple commas in fields.

Some may propose this approach:
props.conf

[your_sourcetype]
REPORT-blah = hash_delim

transforms.conf

[hash_delim]
DELIMS = "#"
FIELDS = field1, field2, field3 etc

However, I cannot find a delim which must not be in any fields.

Please help. Any suggestion?

Also, why would commas inside the double quotation marks will be considered as a delimiter in a csv?

fdi01
Motivator

try this :

[hash_delim]
DELIMS = ","
FIELDS = field1, field2, field3 etc

see this example below, Extract comma-delimited values into fields:

[extract_csv]
DELIMS = ","
FIELDS = "field1", "field2", "field3"

This example assigns the extracted values from _raw to field1, field2 and field3 (in order of extraction). If more than three values are extracted, the values without a matching field name are ignored.

Get Updates on the Splunk Community!

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...