How would I go about parsing out/extracting the field data for the following log format?
"fieldname1":"fieldvalue1","timestamp":"2017-02-07 14:19:54.166","ip_address":"3.3.3.3","user_id":"USER1"
"fieldname1":"fieldvalue1","timestamp":"2017-02-07 14:19:52.395","fieldname2":"fieldvalue2","user_id":"USER2"
"fieldname1":"fieldvalue1","timestamp":"2017-02-07 14:19:50.316","ip_address":"8.8.8.8","fieldname2":"fieldvalue2"
Not all lines of logs will contain all the same fields, but field names are constant. The fields are always comma separated and then in "field":"value" pairs. Currently, I have separate field extractions for each interesting field such as:
\"fieldname1\":\"(?P[a-zA-z]*)
Is this the right way to do this or is there an easier or more proper method?
@nunyabizness123 - Did the answer provided by karlbosanquet help provide a working solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!
Have you had a look at DELIMS in transforms.conf? Here is something that should work;
[comma_colon]
DELIMS = ",", ":"