Hello,
I have a CSV file that has 21 fields, I need to remove fields 1,2 & 15 from the CSV file as this contains sensitive information.
The list is comma separated but I can't work out how to remove these at index time. I have the option to remove or mask but the value in each field is of different length.
Can anyone help with the props & transforms required for this please.
Regards
Rob
After several regex suicide commitments, I've got this below working:
SEDCMD-subst1 = s/(([^;]*;){NN})[^;]*;(.*)/\1SUBSTITUTION;\3/g
You can replace the semicolon with your delimiter, NN is an N-1 field to replace, SUBSTITUTION is the replacement text. You can have an empty string there to remove the field.
Try using SEDCMD. This is the method often used for masking, but can also omit fields. You may need to adjust the regex to match your data.
In props.conf:
[mySourcetype]
SEDCMD = s/([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),([^,]*)/\3,\4,\5,\6,\7,\8,\9,\10,\11,\12,\13,\14,\16,\17,\18,\19,\20,\21/