Getting Data In

CSV file with last 2 fields XML payloads

odigokid
Engager

Need help with the following CSV (everything I am trying, the XML fields are getting parsed incorrectly)

so I have a CSV file with a header line and then data record

The last two fields - FullRequest, and FullResponse - are SOAP payloads which have \n and ',' in the payload - so splunk is treating the newline as a new event, and it's also chopping at the comma because that's the delimiter.

The other fields before these are what I would call your standard CSV fields in "","","","" - but as you can see some fields can be empty (i.e. ,"",)

so looking for approaches to parsing this log file.

0 Karma

woodcock
Esteemed Legend

I generally use INDEXED_EXTRACTIONS which should work fine for your data:
http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Extractfieldsfromfileswithstructureddata

0 Karma

odigokid
Engager

Hi - this is my current props.conf which is not working

DATETIME_CONFIG =
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

Image of what I am seeing on search head - the xml is getting broken on the newlines

alt text

0 Karma

odigokid
Engager

Image link - link text

0 Karma

odigokid
Engager
0 Karma

ssadanala1
Contributor

posting a sample event will help

0 Karma

odigokid
Engager

I tried to attach but stated I don't have enough karma points - let me paste here. (I have not put all the data in the payloads due to customer data - but I have put a line there that has , in the data. and you see the "newline's" in the payloads.

LogType(v1.0),RootLogId,SubLogId,TransactionId,Instance,Operation,Status,User,Hostname,Protocol,Target,StartTime,ExecuteTime,ResponseCode,FullRequest,FullResponse
"southbound","PLP1EM01PL61804231005392658CAI3G1_2","/1/1/1","","","PGW_Create","SUCCESSFUL","","PLP1EM01PL6","SOAP","PGW-SNQ","2018-04-23 10.05.39.892","00 00:00:00.843","0","

 #### more tags and data - data can have comma's (ex. below)
<serviceType>serviceTypeId=0,OU=SERVICE,OU=UMA,NE=MOBILE_DATA_SERVER</serviceType>

",

 #### more tags and data - data can have comma's (ex. below)
<serviceType>serviceTypeId=0,OU=SERVICE,OU=UMA,NE=MOBILE_DATA_SERVER</serviceType>

"

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...