Getting Data In

How to Parsing Apache Access Log ?

shoautorola
New Member

Hi,

I have apache access log with this pattern:

%h %t '%r' '%q' %s %b %D %S %U %v %{User-Agent}i

{text:ip} [{date:Date,dd/MMM/yyyy:HH:mm:ss z}] '{string:HTTP Request Method' '{string:Query String}' {number:Response Code} {string:Bytes Sent} {number:Request Processing Time} {string:User Session ID} {string:Requested URL Path} {string:Local Server Name} {string:User Agent}

i'm struggling to parse the data using regular expression and the overall process how to parse the file.

Here is an example data from the log:
88.117.159.10 [22/Jan/2013:10:57:21 +0100] 'GET /dealers/actions.do HTTP/1.1' '' 200 69068 56 64EB37C2102324BD81E0E0B93243C2232 /dealers/actions.do www.simpledns.com Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0

any kind of help will be appreciated.

Tags (1)
0 Karma

yannK
Splunk Employee
Splunk Employee

Create a specific sourcetype for your data, and define a search time field definition.
see http://docs.splunk.com/Documentation/Splunk/5.0.1/Admin/Transformsconf

example:

in props.conf

[myapache]
REPORT-extract_myapache= extract_myapache

and in transforms.conf

[extract_myapache]
DELIMS = " "
FIELDS = "field1", "field2", "field3"

0 Karma

sdaniels
Splunk Employee
Splunk Employee

Have you tried using the default sourcetypes for access logs? If you assign the sourcetype of access_combined for example Splunk should automatically create the fields for you. Just change the sourcetype in your inputs.conf setting for that file monitor and restart splunk.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Listofpretrainedsourcetypes

In case you run into any issues there are a lot of other postings you can look at here to see if one can help you as well:

http://splunk-base.splunk.com/search/?q=apache&Submit=Search

shoautorola
New Member

it's much easier to parse custom logs in XpoLog Center using simple wizard. Why not Splunk provide that? I'm evaluation splunk as one of our potential log management system to adopt in our company but seems it lack of some functionality that can make your life easier.

0 Karma

shoautorola
New Member

it can't parse when i choose existing source type access_combined except the timestamp. It can detect the time stamp only.

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...