Hello,
I am trying to get control with the cisco Mars logs, and have trouble with the separator. Acording the manual, event should look like :
33750»Wed Jul 27 16:16:06 PDT 2005»BR-FW-1»10.1.2.4»9000»10.1.5.20»80»6»<134>Jan 06 2003
11:03:53: %PIX-6-302001: Built inbound TCP connection 21000 for faddr 10.1.2.4/9000 gaddr
10.1.5.20/80 laddr 10.1.5.20/80
Default splunk makes :
12/05/2010 05:41:55.000 351649?Wed May 12 05:41:55 CEST 2010?vm37.dce.local?0.0.0.0?0?10.75.0.37?0?-1?<30>May 12 03:51:55 sfcb[19464844]: --- Caching ClassProvider for /var/lib/sfcb/registration/repository/vmware/esxv2/classSchemas (1.0-3) using 448 bytes
So there is a mismatch between character set. I tried
[mars] CHARSET=ISO-8859-1
but get
2:47:10.000 AM
223825�Thu May 06 02:47:10 CEST 2010�oc-pix515.tc.oc.local�10.75.25.45�3049�194.109.22.18�6666�6�<164>May 06 2010 04:20:12: %PIX-4-106023: Deny tcp src inside:10.75.25.45/3049 dst internet:194.109.22.18/6666 by access-group "internet-out" [0x329cf230, 0x0]
Anyone familiar with CS-mars?
Thanks
I have the following in my props.conf for my CS MARS archive files, and it works for me:
[cisco_mars_rm]
TIME_PREFIX = ^\d+\\xFF
SHOULD_LINEMERGE = true
MUST_BREAK_AFTER = \\xFF\\xFF
The Cisco MARS raw message add-on was posted here: http://www.splunkbase.com/apps/All/4.x/Add-On/app:Cisco+MARS+Archive+Add-on
Hi, I will be posting a Cisco Mars add-on shortly. In the mean time here are a few things ive worked out.
This is only going to work with the raw message logs for now. i.e. rm-6050-605-1273214234_2010-05-07-06-11-44_2010-05-07-06-40-00
In transforms I find these helpful.
[cisco_mars_rm]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype
REGEX = (rm-)
FORMAT = sourcetype::cisco_mars_rm
[cisco_mars_syslog]
DEST_KEY = MetaData:Sourcetype
REGEX = (%MARS)
FORMAT = sourcetype::cisco_mars_syslog
[cisco_mars_device_name]
REGEX = \d+\\x\w{2}\S+\s\S+\\xFF(\S+)\\x
FORMAT = dvc_name::$1
[mars_attacker]
REGEX = <sd:attacker><sd:addr cid:locality=\"\S+\">(\S+)</sd:addr>
FORMAT = attacker::$1
[mars_target]
REGEX = <sd:target><sd:addr cid:locality=\"\S+\">(\S+)</sd:addr>
FORMAT = target::$1
In props:
TRANSFORMS-syslog = cisco_mars_syslog, cisco_mars_rm
[cisco_mars_rm]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_PREFIX = \d+\\x\w{2}
TIME_FORMAT = %m/%d/%Y %H:%MS
REPORT-dvc = cisco_mars_device_name
REPORT-attacker = mars_attacker,mars_target
FIELDALIAS-srcip = attacker AS src_ip target AS dest_ip
[cisco_mars_syslog]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_PREFIX = \d+\\x\w{2}
TIME_FORMAT = %m/%d/%Y %H:%MS
REPORT-dvc = cisco_mars_device_name
For the IPS logs I find this searches to be useful:
[Cisco MARS Archive - IPS Alerts]
dispatch.earliest_time = -24h
dispatch.latest_time = +0s
displayview = flashtimeline
search = sourcetype::cisco_mars_rm | xmlkv
Having recently battled MARS logs, I empathize with you.
The delimiter is a hex BB (decimal 187). I overcame it by replacing with a ~ via SED:
[source::mars-logs] SEDCMD-delims = s/\\xBB/~/g
and then building my field extraction rules utilizing the ~ delimiter.
Hope this helps.
Sorry for the late reply - SEDCMD processes at indexing, so once you've indexed it's too late. Try reprocessing your source log files to see if the above works for you.
Hai Jeff,
I've added
[cisco_mars]
SEDCMD-delims = s/\xBB/~/g
Should this replace the characters after indexing? no results here.
Thanks,
Hai Jeff,
Will test the sed change, if this works I am oke!. not sure what gkanapathy means exactly,,cause the events are nice and gently breaked with timestamps
You might be able to also solve this with the Splunk SHOULD_LINEMERGE = false
and LINE_BREAKER
settings. I'm not that familiar with the file format, so I can't be much more specific than this.