Splunk Search

Error with regex for syslog event

kseshadri
New Member

Running splunk on windows2003. I am getting the events but it seems my regex is not working right on the event.

Sample syslog event:

Feb 10 13:40:24 066.082.009.081 TPS,1,HN-IP,67.143.66.109,PC-IP,67.143.66.110,OUT,0,IN,0,update.intervideo.com:80,http://update.intervideo.com/Ping.asp

should be converted to the following fields

FORMAT = host::$1 record_type::$2 hn_ip::$3 pc_ip::$4 bytes_out::$5 bytes_in::$6 domain::$7

However it is not working.. could someone give me a clue as to whats wrong with the regex ?

REGEX = ^.*[^\:]*\d\d\:\d\d\:\d\d[^\:]*?\s(\d+\.\d+\.\d+\.\d+)\s[^,]+,([^,]+),HN-IP,([^,]+),PC-IP,([^,]+),OUT,([^,]+),IN,([^,]+),([^,]+),[[bc_uri]]

[Hughes_TPS]
#Feb 10 13:40:24 066.082.009.081 TPS,1,HN-IP,67.143.66.109,PC-IP,67.143.66.110,OUT,0,IN,0,update.intervideo.com:80,http://update.intervideo.com/Ping.asp
REGEX = ^.*[^\:]*\d\d\:\d\d\:\d\d[^\:]*?\s(\d+\.\d+\.\d+\.\d+)\s[^,]+,([^,]+),HN-IP,([^,]+),PC-IP,([^,]+),OUT,([^,]+),IN,([^,]+),([^,]+),[[bc_uri]]
FORMAT = host::$1 record_type::$2 hn_ip::$3 pc_ip::$4 bytes_out::$5 bytes_in::$6 domain::$7

inputs.conf

[udp://514]
sourcetype = syslog
no_appending_timestamp = true

[host::192.168.10:81]
sourcetype = HughesTPS
KV_MODE = none
REPORT-syslog = Hughes_TPS
SEGMENTATION = inner
Tags (1)
0 Karma

Lowell
Super Champion

I'm not sure you can mix-and match numbered group-based extraction with splunk's advanced regex (e.g. [[bc_url]])

Perhaps this would work if you used named groups. You could try using a tranforms.props like this:

BTW, you don't want to override the "host" field like this. If you want to explicitly set the "host" value that get's indexed, you should do it with a special index-time transformer.

[Hughes_TPS_host]
#Feb 10 13:40:24 066.082.009.081
DEST_KEY = MetaData:Host
REGEX    = ^.*[^\:]*\d\d\:\d\d\:\d\d[^\:]*?\s(\d+\.\d+\.\d+\.\d+)\s
FORMAT   = host::$1

[Hughes_TPS]
#Feb 10 13:40:24 066.082.009.081 TPS,1,HN-IP,67.143.66.109,PC-IP,67.143.66.110,OUT,0,IN,0,update.intervideo.com:80,http://update.intervideo.com/Ping.asp
REGEX = ^.*[^\:]*\d\d\:\d\d\:\d\d[^\:]*?\s\d+\.\d+\.\d+\.\d+\s[^,]+,(?P<record_type>[^,]+),HN-IP,(?P<hn_ip>[^,]+),PC-IP,(?P<pc_ip>[^,]+),OUT,(?P<bytes_out>[^,]+),IN,(?P<bytes_in>[^,]+),(?P<domain>[^,]+),(?P<url>.+)

If you get the entry above to work, then try using the fancy combo splunk regex thing....

[Hughes_TPS]
#Feb 10 13:40:24 066.082.009.081 TPS,1,HN-IP,67.143.66.109,PC-IP,67.143.66.110,OUT,0,IN,0,update.intervideo.com:80,http://update.intervideo.com/Ping.asp
REGEX = ^.*[^\:]*\d\d\:\d\d\:\d\d[^\:]*?\s\d+\.\d+\.\d+\.\d+\s[^,]+,(?P<record_type>[^,]+),HN-IP,(?P<hn_ip>[^,]+),PC-IP,(?P<pc_ip>[^,]+),OUT,(?P<bytes_out>[^,]+),IN,(?P<bytes_in>[^,]+),(?P<domain>[^,]+),[[bc_uri]]

Your props.conf should probably look something like this:

[host::192.168.10]
sourcetype = HughesTPS

[HughesTPS]
KV_MODE = none
TRANSFORM-host = Hughes_TPS_host
REPORT-syslog = Hughes_TPS
# I wouldn't recommend messing with segmentation unless you're and advanced user.
# SEGMENTATION = inner
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...