I am running pfSense in my environment. Currently, I am sending logs through UDP 50000, and my source type is pfsense-firewall. My props.conf and transforms.conf look like this:
props.conf
[source::udp:50000]
TRANSFORMS-pfsense-firewall = pfsense-firewall
SHOULD_LINEMERGE = true
TRUNCATE = 0
MUST_NOT_BREAK_AFTER = pf: .* rule ([-\d]+\/\d+)\(.*?\):
MUST_BREAK_AFTER = pf: .* (<|>) +(\d+\.\d+\.\d+\.\d+)\.?(\d*)\:
REPORT-pfsense-firewall = pfsense-firewall
transforms.conf
[pfsense-firewall]
REGEX = .* (?pass|block) .* (?TCP|UDP|IGMP|ICMP) .* (?(\d+\.\d+\.\d+\.\d+))\.?(?(\d*)) [<|>] (?(\d+\.\d+\.\d+\.\d+))\.?(?(\d*)): (.*)
CLEAN_KEYS = 1
MV_ADD = 0
I am also currently sending virus logs from havp (pfSense antivirus package) through the same port. The logs from pfSense are multi-lined, but the logs from havp are single-lined.
A log sample from havp:
Oct 15 10:22:28 192.168.89.254 Oct 15 10:22:34 havp[2937]: 192.168.2.3 GET 200 http://www.eicar.org/download/eicar.com 380+68 VIRUS Clamd: Eicar-Test-Signature
How can I separate the logs to their respective source types based on the regex? Sorry, I'm pretty new to Splunk, so this may come across as a newbie question.
You could do something like this (untested, so double-check for typos before copypastaing):
props.conf:
[source::udp:50000]
sourcetype = havp
TRANSFORMS-set_pfsense_sourcetype = set_pfsense_sourcetype
...
transforms.conf:
[set_pfsense_sourcetype]
REGEX = [\r\n]
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::pfsense-firewall
...
The sourcetype will default to hvap as set in props.conf, and will be overwritten to pfsense in transforms.conf if a line break is present in the event.
Note, I'd personally prefer giving each source its own UDP port... but this should work as well.
You could do something like this (untested, so double-check for typos before copypastaing):
props.conf:
[source::udp:50000]
sourcetype = havp
TRANSFORMS-set_pfsense_sourcetype = set_pfsense_sourcetype
...
transforms.conf:
[set_pfsense_sourcetype]
REGEX = [\r\n]
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::pfsense-firewall
...
The sourcetype will default to hvap as set in props.conf, and will be overwritten to pfsense in transforms.conf if a line break is present in the event.
Note, I'd personally prefer giving each source its own UDP port... but this should work as well.
Unfortunately, havp do not have the settings to send logs on a different port.
I'll give it a try!
in props.conf make 2 stanzaz for pfsense and hvap
[pfsense_sourcetype]
.....
REPORT-pfsense-firewall = pfsense-firewall
[hvap_sourcetype]
....
REPORT-hvap-firewall = hvap-firewall
In transforms.conf
[pfsense-firewall]
REGEX = .* (?pass|block) .* (?TCP|UDP|IGMP|ICMP) .* (?(\d+\.\d+\.\d+\.\d+))\.?(?(\d*)) [<|>] (?(\d+\.\d+\.\d+\.\d+))\.?(?(\d*)): (.*)
CLEAN_KEYS = 1
MV_ADD = 0
[ hvap-firewall]
I can make two stanzas even though my source is udp:50000?
I defined UDP port 50000 as pfsense-firewall in my data inputs.