All,
Can I have a quick sanity check on this transforms.conf? Basically I want to keep any log which has fatal, critical and error and drop the rest. THEN turn around and drop some of those logs which are "known".
# transforms.conf
[XYZ_keep]
REGEX=(?!error)|(?!fatal)|(?!crit)|(?!splunk)
DEST_KEY=queue
FORMAT=nullQueue
[XYZ_nullqueue]
REGEX=priority=INFO app_name=SolrCloud|location=MissingListingAttributeList|JmxHealthCheck
DEST_KEY=queue
FORMAT=nullQueue
You need something like this in props.conf:
TRANSFORMS-this_string_does_not_matter = XYZ_null_all, XYZ_keep, XYZ_null_some
Then in transforms.conf:
[XYZ_null_all]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[XYZ_keep]
REGEX=(?!error)|(?!fatal)|(?!crit)|(?!splunk)
DEST_KEY=queue
FORMAT=indexQueue
[XYZ_null_some]
REGEX=priority=INFO app_name=SolrCloud|location=MissingListingAttributeList|JmxHealthCheck
DEST_KEY=queue
FORMAT=nullQueue
This will NOT work, it will send everything to nullQueue
.
You need something like this in props.conf:
TRANSFORMS-this_string_does_not_matter = XYZ_null_all, XYZ_keep, XYZ_null_some
Then in transforms.conf:
[XYZ_null_all]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[XYZ_keep]
REGEX=(?!error)|(?!fatal)|(?!crit)|(?!splunk)
DEST_KEY=queue
FORMAT=indexQueue
[XYZ_null_some]
REGEX=priority=INFO app_name=SolrCloud|location=MissingListingAttributeList|JmxHealthCheck
DEST_KEY=queue
FORMAT=nullQueue
I recommend avoiding the double-negative logic in XYZ_keep to prevent mis-understandings by future maintainers. I added the case-insensitive flag to allow for "Error", "ERROR", and "error".
[XYZ_keep]
REGEX=(?i)(error)|(fatal)|(crit)|(splunk)
DEST_KEY=queue
FORMAT=indexQueue