We're ingesting structured JSON logs from a source and would like to run the equivalent of the extract command on one of the event's sub fields. The events look something like this:
{
"field1":"value1",
"field2":"value2",
"field3":"value3",
"msg":"field4=value4 field5=value5 field6=value6"
}
The top level field1/field2/field3/msg fields are all being extracted as expected. However, we'd also like to extract arbitrary key/value pairs defined in the msg field, ideally at index time so that they're available to all searches. The key/value pairs that exist in the msg field are not known beforehand. Is it possible to still extract them at index time and make them available to searches?
We've been able to achieve the desired result with a search command chain like the following:
...base search...
| rename _raw AS _temp
| rename msg AS _raw
| extract pairdelim="?&" kvdelim="="
| rename _raw AS msg
| rename _temp AS _raw
However, we have some dashboards that run lots of searches, and we don't want to hack the above command chain into every individual search query.
... View more