Running splunk on windows2003. I am getting the events but it seems my regex is not working right on the event.
Sample syslog event:
Feb 10 13:40:24 066.082.009.081 TPS,1,HN-IP,67.143.66.109,PC-IP,67.143.66.110,OUT,0,IN,0,update.intervideo.com:80,http://update.intervideo.com/Ping.asp
should be converted to the following fields
FORMAT = host::$1 record_type::$2 hn_ip::$3 pc_ip::$4 bytes_out::$5 bytes_in::$6 domain::$7
However it is not working.. could someone give me a clue as to whats wrong with the regex ?
REGEX = ^.*[^\:]*\d\d\:\d\d\:\d\d[^\:]*?\s(\d+\.\d+\.\d+\.\d+)\s[^,]+,([^,]+),HN-IP,([^,]+),PC-IP,([^,]+),OUT,([^,]+),IN,([^,]+),([^,]+),[[bc_uri]]
[Hughes_TPS]
#Feb 10 13:40:24 066.082.009.081 TPS,1,HN-IP,67.143.66.109,PC-IP,67.143.66.110,OUT,0,IN,0,update.intervideo.com:80,http://update.intervideo.com/Ping.asp
REGEX = ^.*[^\:]*\d\d\:\d\d\:\d\d[^\:]*?\s(\d+\.\d+\.\d+\.\d+)\s[^,]+,([^,]+),HN-IP,([^,]+),PC-IP,([^,]+),OUT,([^,]+),IN,([^,]+),([^,]+),[[bc_uri]]
FORMAT = host::$1 record_type::$2 hn_ip::$3 pc_ip::$4 bytes_out::$5 bytes_in::$6 domain::$7
inputs.conf
[udp://514]
sourcetype = syslog
no_appending_timestamp = true
[host::192.168.10:81]
sourcetype = HughesTPS
KV_MODE = none
REPORT-syslog = Hughes_TPS
SEGMENTATION = inner
I'm not sure you can mix-and match numbered group-based extraction with splunk's advanced regex (e.g. [[bc_url]]
)
Perhaps this would work if you used named groups. You could try using a tranforms.props
like this:
BTW, you don't want to override the "host" field like this. If you want to explicitly set the "host" value that get's indexed, you should do it with a special index-time transformer.
[Hughes_TPS_host]
#Feb 10 13:40:24 066.082.009.081
DEST_KEY = MetaData:Host
REGEX = ^.*[^\:]*\d\d\:\d\d\:\d\d[^\:]*?\s(\d+\.\d+\.\d+\.\d+)\s
FORMAT = host::$1
[Hughes_TPS]
#Feb 10 13:40:24 066.082.009.081 TPS,1,HN-IP,67.143.66.109,PC-IP,67.143.66.110,OUT,0,IN,0,update.intervideo.com:80,http://update.intervideo.com/Ping.asp
REGEX = ^.*[^\:]*\d\d\:\d\d\:\d\d[^\:]*?\s\d+\.\d+\.\d+\.\d+\s[^,]+,(?P<record_type>[^,]+),HN-IP,(?P<hn_ip>[^,]+),PC-IP,(?P<pc_ip>[^,]+),OUT,(?P<bytes_out>[^,]+),IN,(?P<bytes_in>[^,]+),(?P<domain>[^,]+),(?P<url>.+)
If you get the entry above to work, then try using the fancy combo splunk regex thing....
[Hughes_TPS]
#Feb 10 13:40:24 066.082.009.081 TPS,1,HN-IP,67.143.66.109,PC-IP,67.143.66.110,OUT,0,IN,0,update.intervideo.com:80,http://update.intervideo.com/Ping.asp
REGEX = ^.*[^\:]*\d\d\:\d\d\:\d\d[^\:]*?\s\d+\.\d+\.\d+\.\d+\s[^,]+,(?P<record_type>[^,]+),HN-IP,(?P<hn_ip>[^,]+),PC-IP,(?P<pc_ip>[^,]+),OUT,(?P<bytes_out>[^,]+),IN,(?P<bytes_in>[^,]+),(?P<domain>[^,]+),[[bc_uri]]
Your props.conf
should probably look something like this:
[host::192.168.10]
sourcetype = HughesTPS
[HughesTPS]
KV_MODE = none
TRANSFORM-host = Hughes_TPS_host
REPORT-syslog = Hughes_TPS
# I wouldn't recommend messing with segmentation unless you're and advanced user.
# SEGMENTATION = inner