Getting Data In

transforms.conf

shangshin
Builder
[my_fields]
REGEX = ^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user_id]]\s++[[sbstring:req_time]]\s++[[qstring:method_url_protocol]]\s++[[nspaces:status]]\s++[[nspaces:bytes]]\s++[[qstring:referer_url]]\s++[[qstring:useragent]]

[method_url_protocol]
DELIMS = " "
FIELDS = method, url, protocol

Hi,
I define these 2 stanzas above in transforms.conf and expect to extract some info from web access log. As you can see the sample quoted string below, it contains 3 fields. However, these 3 fields are not extracted our successsfully. Can you shed some light on it?

"POST /amazon.com/view.do HTTP/1.1"

Tags (1)

jpmasseria
Engager

Try using [[access-request]] to extract method,uri and version.

Below is what I used and it worked for me.

REGEX =
^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user_id]]\s++[[sbstring:req_time]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:bytes]]\s++[[qstring:referer_url]]\s++[[qstring:useragent]]\s++[[qstring:someurl]]\s++[[nspaces:response_time]]

richprescott
Path Finder

I think Ayn is referring to adding the regex as follows in props.conf.


[access_log_reg]
NO_BINARY_CHECK = 1
pulldown_type = 1
REPORT-myfields = my_fields,method_url_protocol

0 Karma

Ayn
Legend

This is why I'm asking if you're actually referring to that transform from props.conf. If you just setup the transform but don't refer to it anywhere, it won't ever be applied.

0 Karma

shangshin
Builder

here is the complete sample roq in web access log
10.39.208.2 - clinet_user_id [29/May/2012:14:04:10 -0400] "POST /amazon.com/view.do HTTP/1.1" 200 1214 "google.com" "Java/1.5.0_06"

As you can, the field of method, url and protocol can be extracted out as a single value using the first stanza (my_fields). However, the second stanza (method_url_protocol) is unable to parse the value. I guess I didn't set it up properly...

0 Karma

Ayn
Legend

What about the method_url_protocol transform? That's the one that, if configured properly, would do the work.

0 Karma

shangshin
Builder

yes, that's right. In props.conf,

[access_log_reg]
NO_BINARY_CHECK = 1
pulldown_type = 1
REPORT-myfields = my_fields

0 Karma

Ayn
Legend

Are you referring to these transforms in props.conf?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...