I've got a bunch of loglines that are very boring key value pairs - comma separated list of key=value. (So far, so good.) The first few (let's say three for example purposes) are always the same, and the rest (the number is dynamic) are variable depending on what's in the first three.
For the purpose of reducing the amount of data I'm ingesting, I'd like to just list the values for the first three columns and not identify the keys. That is, I'd take a line that starts out like this:
a=1,b=2,c=3,d=4,e=5,f=6
and simplify it to
1,2,3,d=4,e=5,f=6
(That only saves a few chars in this example, but the full dataset has more verbose field names, so it adds up to a lot!)
I thought I could just write a block in transforms.conf like this:
[foo]
REGEX = ^(?<a>[^,]*),(?<b>[^,]*),(?<c>[^,]*),
and then apply it in props.conf. I've done that but I'm now getting no fields extracted at all. (I should mention that this is an index time extraction, so I'm doing it with TRANSFORMS-foo.)
I'm at a bit of a loss about how to figure out why this isn't working. I kinda expected that this might prevent extraction of the kv pairs past the REGEX, but it seems to now do nothing. Suggestions about how to sort this out?
... View more