How do I do a field extraction with a special character?

[kcchu01]

Hello, I have some logs that required to extract the fields. the raw data is in the format as below.

"xxx","yyy","zzz","sd323ds","3343tds23@343"

I would like to extract the field like this
field 1: xxx
field 2: yyy
field 3: zzz
field 4: sd323ds
field 5: 3343tds23@343

I use the field extractor, and it almost did what I want. The regular expression extracts the field by the double quotes, but there are some cases where double quotes may appear in some field contents, such as

"xxx","yyy","zzz","sd323ds","3343"Hello World" tds23@343"

I want the field 5 is 3343"Hello World" tds23@343, but it cannot be done by the current regular expression.

How can I modify my existing regular expression to achieve this?

[vnravikumar]

Try this

| makeresults 
| eval test= "\"xxx\",\"yyy\",\"zzz\",\"sd323ds\",\"3343\"Hello World\" tds23@343\"" 
| rex field=test "\"(?P<field1>.*)\",\"(?P<field2>.*)\",\"(?P<field3>.*)\",\"(?P<field4>.*)\",\"(?<field5>.*)"

[whrg · Accepted solution]

Hi!
Your logs look very much like a CSV file. How about you add a CSV header to your log file:

field_1,field_2,field_3,field_4,field_5
"xxx6","yyy","zzz","sd323ds","3343tds23@343"
"xxx","yyy","zzz","sd323ds","3343"Hello World" tds23@343"
...

Then you only need to set sourcetype=csv on your input. Splunk will extract the fields automatically.

Alternatively, have a look at this regular expression:

^"(?<field_1>.*)","(?<field_2>.*)","(?<field_3>.*)","(?<field_4>.*)","(?<field_5>.*)"$

Example on regex101.com

[kcchu01]

Thank you very much for your suggestion which almost perfect for my need. I modified a little bit such that the line break is also accepted. For my understanding the expression is separating the fields by the "," . I am asked that if the content is containing this pattern, fields will be extracted wrongly. Is it possible for regular expression to handle such the case that may be something like "escape character"? I am not good at regular expression and have no idea on this.