Hi,
I have a multi line flat file where I want to ignore/drop specifc events. I'm using the Universial Forwarder, so as I understand it, the indexer needs to drop/ignore the event. Below is my props.conf on the indexer. I want to drop/ignore any event that is not an ERROR. I tried the PREAMBLE_REGEX property setting a NOT regex and one explicitly looking for DEBUG. Neither are working.
Sample event to ignore:
02/09/2015 11:37:54,807 - DEBUG - https://Blah
[Alerts]
BREAK_ONLY_BEFORE_DATE = TRUE
SHOULD_LINEMERGE = TRUE
TIME_FORMAT=%m/%d/%Y %T
TRUNCATE = 0
MAX_DAYS_AGO = 2
PREAMBLE_REGEX =^((?!\d\d.\d\d.\d\d\d\d\s\d\d:\d\d:\d\d\,\d+\s\-\sERROR).)*
PREAMBLE_REGEX=^\d\d.\d\d.\d\d\d\d\s\d\d:\d\d:\d\d\,\d+\s\-\sDEBUG
Any suggestions?
Thank you!
Chris
Ah, so [sourcetype::mySourceType] is not correct. I found I just needed [mySourceType] in the props.config, now its working.
Chris
Ah, so [sourcetype::mySourceType] is not correct. I found I just needed [mySourceType] in the props.config, now its working.
Chris
Ugh, this is baffling. Nothing is working. I placed splunkd in debug mod and didnt see anything in the errors. Is there a way to debug if the "transforms" are being hit?
Tx
Chris
Give this a try
transforms.conf (on INDEXER)
[eliminate-debug]
REGEX=(?m)-\s*DEBUG\s*-
DEST_KEY=queue
FORMAT=nullQueue
props.conf (on INDEXER)
[sourcetype::mySourceType]
TRANSFORMS-trash = eliminate-debug
Restart/reload the Indexer after change.
Output of ./splunk cmd btool transforms list below. So it looks like the transforms are loading. Just cant figure out why they it is not working..
[eliminate-debug]
CAN_OPTIMIZE = True
CLEAN_KEYS = True
DEFAULT_VALUE =
DEST_KEY = queue
FORMAT = nullQueue
KEEP_EMPTY_VALS = False
LOOKAHEAD = 4096
MV_ADD = False
REGEX = (?m)-\s*DEBUG\s*-
SOURCE_KEY = _raw
WRITE_META = False
Ok, found examples of using transforms.conf, but its still not working. Below is my transforms.conf. I want to drop DEBUG statements and INFOs and other stuff. Only keep ERRORS
Sample Error:
02/09/2015 16:25:54,220 - ERROR - ECommerceBlah
in transforms.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[setparsing]
REGEX = ERROR
DEST_KEY = queue
FORMAT = indexQueue
In props.conf
[sourcetype::mySourceType]
TRANSFORMS-trash = setnull,setparsing
Any ideas?
Thank you,
Chris