I need to reduce our licensing usage by filtering common, valid, no-news-is-good-news domains out of our Barracuda Web Filter logs. I am trying to do this by sending such log messages to the nullQueue, but I clearly am not doing it correctly.
The set_bwf transform works, but the bwf_discard transform does not.
Given that I am a regex newbie, I suspect that I have crafted the regular expression under "[bwf_discard]" incorrectly.
Here's what I have so far:
props.conf:
[source::udp:514]
TRANSFORMS-discard=bwf_discard
TRANSFORMS-changesourcetype=set_bwf
[syslog:bwf]
REPORT-extract=bwf_extract,user
transforms.conf:
[bwf_discard]
REGEX = (?m)(commondomain1.com|commondomain2.com|commondomain3.com|commondomain4.com|commondomain5.com)
DEST_KEY = queue
FORMAT = nullQueue
[set_bwf]
REGEX = barracuda-hostname.localdomain
FORMAT = sourcetype::syslog:bwf
DEST_KEY = MetaData:Sourcetype
[bwf_extract]
DELIMS = " "
FIELDS = bwf_month, bwf_day, bwf_time, bwf_hostname, bwf_daemon_info, bwf_timestamp, bwf_number_1, bwf_src_ip, bwf_dest_ip, bwf_content_type, bwf_src_ip2, bwf_dest_url, bwf_data_size, bwf_md5_anchor, bwf_action, bwf_reason, bwf_format_version, bwf_match_flag, bwf_TQ_flag, bwf_action_type, bwf_src_type, bwf_src_detail, bwf_dest_type, bwf_dest_detail, bwf_spy_type, bwf_spy_id, bwf_infection_score, bwf_matched_part, bwf_matched_category, bwf_user
[user]
REGEX = ([^\s]+)\s\[([\w\:]+)\]\s+$
FORMAT = category::$1 user::$2
... View more