I've got a log file that get's 2 different event formats depending on if debugging is turned on. When debugging is turned on I don't want the debug events forwarded but I do want the normal events forwarded as normal.
I have a regular expression that will only include my normal events that looks like this: [0-9]*:.*[%].*
I know that I can create a transforms.conf file in $SPLUNK_HOME/etc/apps/appName/local
to filter events.
In inputs.conf
I have the following:
[monitor:///var/log/boot.log]
disabled = false
followTail = 0
index = zod-os
sourcetype = linux_bootlog
I think if I add the following to transforms.conf
it will do what I want:
[linux_bootlog]
REGEX = [0-9]*:.*[%].*
What I'm not 100% sure of is if I need to create a props.conf
file to point to the transform like I've seen in other answers. I don't want to extract any additional fields other than what Splunk appears to automatically be doing. Also, the debug events are multiline but since they don't match the regex I think they will drop automatically.
Does all of that sound like it will work?
Yes, props.conf is what tells the system when to run the transforms in transforms.conf, so there needs to be a stanza there for your source (or however you want to identify the stuff that you are treating this way).
What I would tend to do is route everything to the nullqueue and then route the matching events back.
[source::/var/log/foo]
# Transforms must be applied in this order
# to make sure events are dropped on the
# floor prior to making their way to the
# index processor
TRANSFORMS-set = setnull, setparsing
[setnull]
REGEX=.*
DEST_KEY = queue
FORMAT = nullQueue
[setparsing]
REGEX = [0-9]*:.*[%].*
DEST_KEY = queue
FORMAT = nullQueue
The above is barely changed at all (only the second regex) from this answer... https://answers.splunk.com/answers/96/how-do-i-exclude-some-events-from-being-indexed-by-splunk.html
Yes, props.conf is what tells the system when to run the transforms in transforms.conf, so there needs to be a stanza there for your source (or however you want to identify the stuff that you are treating this way).
What I would tend to do is route everything to the nullqueue and then route the matching events back.
[source::/var/log/foo]
# Transforms must be applied in this order
# to make sure events are dropped on the
# floor prior to making their way to the
# index processor
TRANSFORMS-set = setnull, setparsing
[setnull]
REGEX=.*
DEST_KEY = queue
FORMAT = nullQueue
[setparsing]
REGEX = [0-9]*:.*[%].*
DEST_KEY = queue
FORMAT = nullQueue
The above is barely changed at all (only the second regex) from this answer... https://answers.splunk.com/answers/96/how-do-i-exclude-some-events-from-being-indexed-by-splunk.html
setparsing stanza should be FORMAT = indexQueue
This is exactly the kind of thing I needed. Thank you very much.