Splunk is breaking ids single event into two events, such as:
4/11/16
2:42:46.152 PM
04/11-14:42:46.152985 00:05:00:00:00:00 -> 00:00:00:05:00:01 type:0x800 len:0x222
10.20.30.40:59406 -> 106.120.151.145:80 TCP TTL:52 TOS:0x0 ID:53190 IpLen:20 DgmLen:532 DF
***A**** Seq: 0xBA0195C4 Ack: 0xBB15F92D Win: 0x3E00 TcpLen: 20
[Xref => http://doc.emergingthreats.net/2008500]
host = ISMeta2 source = /var/log/snort/snort.log
4/11/16
2:42:46.000 PM
[**] [1:2008500:6] ET MALWARE Sogoul.com Spyware User-Agent (SogouIMEMiniSetup) [**]
[Classification: A Network Trojan was Detected] [Priority: 1]
host = ISMeta2 source = /var/log/snort/snort.log
Which appears in snort.log as this one event:
[**] [1:2008500:6] ET MALWARE Sogoul.com Spyware User-Agent (SogouIMEMiniSetup) [**]
[Classification: A Network Trojan was Detected] [Priority: 1]
04/11-14:42:46.152985 00:05:00:00:00:00 -> 00:00:00:05:00:01 type:0x800 len:0x222
10.20.30.40:59406 -> 106.120.151.145:80 TCP TTL:52 TOS:0x0 ID:53190 IpLen:20 DgmLen:532 DF
***A**** Seq: 0xBA0195C4 Ack: 0xBB15F92D Win: 0x3E00 TcpLen: 20
[Xref => http://doc.emergingthreats.net/2008500]
All events start with [**]
I have props.conf configured as follows where snort_alert_full is sourcetype, but that doesn't fix my issue
[snort_alert_full]
BREAK_ONLY_BEFORE = [**]
Thanks in advance
BREAK_ONLY_BEFORE expects a regular expression, [**]
would be extremely malformed for your data. Have you tried \[\*\*\]
?
That being said, you'd be better off utilizing LINE_BREAKER like this:
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\[\*\*\]
TIME_PREFIX = ^([^\r\n]+[\r\n]+){2}
MAX_TIMESTAMP_LOOKAHEAD = 25
TIME_FORMAT = %m/%d-%H:%M:%S.%6N
That'll be MUCH faster and should achieve exactly what you need.
These props.conf settings need to be where the parsing phase happens, that'll be a heavy forwarder or the indexers in this case. The universal forwarder will ignore these.
Hi, I think your answer will definitely work but I don't think I have the props.conf in the right place basically the logs are being forwarded to splunk through universal forwarder. Under etc/deployment-apps I see a folder SplunkUFSnortconfig folder which is where the inputs.conf is configured as to which location to monitor name of index,sourcetype etc. I created props.conf in SplunkUFSnortconfig -> local but that doesn't seem to do anything at all.
Some more sample events:
Event 1
[**] [1:2520169:2541] ET TOR Known Tor Exit Node UDP Traffic group 85 [**]
[Classification: Misc Attack] [Priority: 2]
04/12-11:22:01.201114 00:00:00:05:00:01 -> 00:05:00:00:00:00 type:0x800 len:0x53
93.158.215.174:23320 -> 90.80.70.60:53 UDP TTL:52 TOS:0x28 ID:43018 IpLen:20 DgmLen:69
Len: 41
[Xref => http://doc.emergingthreats.net/bin/view/Main/TorRules]
Event 2
[**] [1:2016104:3] ET TROJAN DNS Reply for unallocated address space - Potentially Malicious 1.1.1.0/24 [**]
[Classification: A Network Trojan was Detected] [Priority: 1]
04/12-11:30:31.770737 00:00:00:05:00:01 -> 00:05:00:00:00:00 type:0x800 len:0x69
125.39.136.74:53 -> 30.40.50.60:56038 UDP TTL:51 TOS:0x0 ID:35728 IpLen:20 DgmLen:91
Len: 63
and my current props.conf because I see [**}
at 2 places so I expanded on your suggestion.
[snort_alert_full]
SHOULD_LINEMERGE = false
LINE_BREAKER =[**]\s+[\d+:\d+:\d+]
That LINE_BREAKER cannot work. First, it doesn't have a capturing group for the linebreaker processor to consume and second, you're not escaping the regex special chars asterisk and square bracket.
Thanks for the help, this did not work, I think there is more to the way the logs are setup as well. For instance - I want splunk to look at event starting with [**]
and then grab the timestamp from the third line of the log. How do I achieve that ?
[**] [1:2008500:6] ET MALWARE Sogoul.com Spyware User-Agent (SogouIMEMiniSetup) [**]
[Classification: A Network Trojan was Detected] [Priority: 1]
04/11-14:42:46.152985 00:05:00:00:00:00 -> 00:00:00:05:00:01 type:0x800 len:0x222
10.20.30.40:59406 -> 106.120.151.145:80 TCP TTL:52 TOS:0x0 ID:53190 IpLen:20 DgmLen:532 DF
***A**** Seq: 0xBA0195C4 Ack: 0xBB15F92D Win: 0x3E00 TcpLen: 20
[Xref => http://doc.emergingthreats.net/2008500]
Thanks in advance.