Getting Data In

How to collect Apache logs in Splunk without a forwarder?

sloshburch
Splunk Employee
Splunk Employee

Someone just asked me an interesting question that I don't have the answer to...but I bet this community does 😉

Has anyone ever created an interesting way of getting apache logs off a server without actually installing a forwarder?

If it requires installing or running any script, then we might as well use a forwarder. But you never know if someone out there found some native way to send this log data into splunk using some other means than the forwarder.

Thanks for any ideas!

dhuseau
Engager

I have this issue too, but I am working on a python script that will send events to Splunk using the HTTP Event Collector. It's not finished yet but, I should get an initial version done this week. When it's done you can call the script via a cron job and it will read the latest logs and send them over to Splunk.

I have the first few commits here: https://github.com/alecdhuse/VPS-Log-Watch

0 Karma

sloshburch
Splunk Employee
Splunk Employee

I like that you're using the HTTP Event Collector but I've been burned before with custom scripts. If I was able to have anything run on the endpoint apache server, I would go with a forwarder (proven, small, "real-time"). Are you able to share the circumstances that motivated this effort?

0 Karma

dhuseau
Engager

I have a website on a shared VPS server. I do not manage the server and I do not have rights to install any software including the Splunk forwarder. However, I am allowed to run scripts and schedule cron jobs.

This seemed like an acceptable way to fill this gap. I am open to other methods of moving the data though.

0 Karma

sloshburch
Splunk Employee
Splunk Employee

Oh ok. I have a similar situation and so I used the tar (not rpm) version to place the splunk forwarder. Then I have a nightly cronjob that simply runs ./splunk start to makes sure it didn't get killed. So far so good and more stable than my poor programming.

I figured I'd share in case my similar scenario inspires something.

0 Karma

dhuseau
Engager

Thanks, that is probably a better solution.

0 Karma

woodcock
Esteemed Legend
0 Karma

hylam
Contributor

NFS (Network File System)

0 Karma

mreynov_splunk
Splunk Employee
Splunk Employee

or you can figure out how to use the fancy new HTTP Event Collector: http://blogs.splunk.com/2015/10/06/http-event-collector-your-direct-event-pipe-to-splunk-6-3/

0 Karma

ltrand
Contributor

You can syslog the data out
or
You can have some kind of scp/scripted pull action from some centralized entity
or
you can write them to a local SQL/DB instance and use dbconnect to pull it
or
you can install the forwarder
:)

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...