We are having difficulty getting exclusions of logs that have fields in Camelcase or have entries that have special characters related to OTEL logs. Fields without capitalization and/or special character values are able to be parsed out, but not others.
Here is an example log that we are looking at (attached as yaml and key portion).
filelog/kube-apiserver-audit-log:
include:
- /var/log/kubernetes/kube-apiserver.log
include_file_name: false
include_file_path: true
operators:
- id: extract-audit-group
type: regex_parser
regex: '\s*\"resourceGroup\"\s*\:\s*\"(?P<extracted_group>[^\"]+)\"\s*'
- id: filter-group
type: filter
expr: 'attributes.extracted_beta == "batch"'
- id: remove-extracted-group
type: remove
field: attributes.extracted_group
- id: extract-audit-api
type: regex_parser
regex: '\"level\"\:\"(?P<extracted_audit_beta>[^\"]+)\"'
- id: filter-api
type: filter
expr: 'attributes.extracted_audit_beta == "Metadata"'
- id: remove-extracted-api
type: remove
field: attributes.extracted_api
- id: extract-audit-verb
type: regex_parser
regex: '\"verb\"\:\"(?P<extracted_verb>[^\"]+)\"'
- id: filter-verb
type: filter
expr: 'attributes.extracted_verb == "watch" || attributes.extracted_verb == "list"'
- id: remove-extracted-verb
type: remove
field: attributes.extracted_verb
The resourceGroup field is compared to something else and failing, verb and level are succeeding.
Here is an example log that would be pulled in.
{"apiVersion":"batch/v1","component":"sync-agent","eventType":"MODIFIED","kind":"CronJob","level":"info","msg":"sent event","name":"agentupdater-workload","namespace":"vmware-system-tmc","resourceGroup":"batch","resourceType":"cronjobs","resourceVersion":"v1","time":"2024-03-14T18:17:11Z"}
... View more
