Getting Data In

How can I import Apache log files?

lonwinters
New Member

A certain web host stopped offering AWStats with its hosting. Instead, they point you to Google Webmaster Tools. I used that for a while, until it stopped reporting the basic stats - unique visitors and hits. Google Analytics, as well as most other stats programs I've looked at are very complex and more than I need. I simply want to do the following, please provide directions. I installed Splunk, started it via the command line and logged in.

  1. Download the Apache log files from the web host
  2. Import log files into Splunk and view the reports.

And that's it. I chose Splunk because I wanted something I could run locally and not have to upload or install anything to the website. Thank you.

Tags (1)
0 Karma

grijhwani
Motivator

First you need some means of exporting the log files from the host to your local drive, then when you have sample files on your drive you need to go to Manager » Data inputs, click "Add data", select "From files and directories", then follow the guided dialogue.

lguinn2
Legend

You should go through the Splunk tutorial and also follow @yannK's advice

yannK
Splunk Employee
Splunk Employee

If you can access the server, install the forwarder on the server where the logs are.
Then setup the input to monitor the folders of the log file
and setup the forwarder to send the logs to the indexer (configured to accept remote data)

Otherwise, you will have to create a script to download the logs to the indexer and monitor them on the indexer.

Hint, for the apache logs, you probably want to specify the sourcetype=access_combined at the input level.

yannK
Splunk Employee
Splunk Employee

Look at @grijhwani 's answer.
Retrieve the log files, and put them in a folder that the indexer is monitoring.
Use the manager UI to create this monitor on the folder and specify the appropriate sourcetype (access_combined)

Then search for the events in Splunk, look for existing apps on splunkbase, or build your own dashboards.
You probably want to a high level report per day, and summarize or accelerate it.

0 Karma

lonwinters
New Member

First of all, I really really appreciate the fast and detailed responses. I did go through some of the documentation, but was hoping to avoid some of the configuration steps. I will go through the tutorial though.

I can download the Apache logs from the server, they are separate files for each day. All I really want to do is import these files into Splunk and view simple reports similar to AW stats. I would be OK if I could only view a day at a time, but it would be nice to look at the data in a monthly view.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...