How do I create a capture group that continues until it matches two exact characters in a row

[rbechtold]

Hey everyone,

This question probably shows my lack of understanding with regex, but this is giving me a headache and it isn't the first time I've run into this problem. I have a raw log that is in this format (with the personal information removed):

...field_one="3-5713-064B-AB34", field_two="45", message="This is a test message: it has no real relevance, but I need it to explain the problem I am facing. The log has both numbers and letters, as well as [brackets], {curly brackets}, Periods..., And many other types of punctuation and characters, such as !@#$%^&*( and ). The other problem is there are also "quotation marks". There are lots of "quotation marks" and commas, in the text itself. Therefore breaking by either quotation marks or commas does not work. This is the end of the test message.", uniquely_named_field="1", field_three="20180105"...

The way I'm attempting to extract the data at the moment looks like this:

\.{3}\w+\=\"(?P[^"]+)\"\,\s\w+\=\"(?P\d+)\"\,\s\w+\=\"(?P[^"]+\"[^"]+\"[^"]+\"[^"]+\"[^"]+)\"\,\s\w+\=\"(?P\d+)\"\,\s\w+\=\"(?P\d+)\"\.{3}

I've tested this in regex101 and it appears to be extracting correctly. However, this will only extract data that needs to be broken exactly at the fifth instance of quotation marks.

I would like to write an extraction that either breaks when it reaches a comma followed by a quotation mark (\,\") or by the uniquely_named_field. However, I can't seem to find a way to do this, and my regex looks terrible.

If anyone could help me out, I would greatly appreciate it.

[niketn]

@rbechtold, based on the details and sample data provided in the question seems like you need to use the extract command.

<YourBaseSearch>
| extract kvdelim="=" pairdelim=","

Following is the run anywhere example based on the sample data provided in the question:

| makeresults
| eval _raw="field_one=\"3-5713-064B-AB34\", field_two=\"45\", message=\"This is a test message: it has no real relevance, but I need it to explain the problem I am facing. The log has both numbers and letters, as well as [brackets], {curly brackets}, Periods..., And many other types of punctuation and characters, such as !@#$%^&*( and ). The other problem is there are also \"quotation marks\". There are lots of \"quotation marks\" and commas, in the text itself. Therefore breaking by either quotation marks or commas does not work. This is the end of the test message.\", uniquely_named_field=\"1\", field_three=\"20180105\""
| extract kvdelim="=" pairdelim=","

[micahkemp]

If you know that your message field is followed by the uniquely_named_field field, you could do:

message="(.*)", uniquely_named_field=

[mayurr98]

i did not understand your question what do you exactly want? what do you want to extract can you give what should be the field name and what should be the value for that field name from the sample event that you gave?

[martin_mueller · Accepted solution]

Here's how you would capture "everything until before quote-comma": message="(.*?)",

Not sure what you mean by "or by the uniquely_named_field", do you have an example event where that would be necessary?