Getting Data In

How can I input data from url

perlish
Communicator

I want to input data from url,such as http://my.site.com/dns_monitor.log
How can I make it ?
Thank you !

Tags (3)
0 Karma

perlish
Communicator

Hi Icrielaa,
I can not access the web server,I can only access the url,and I thought wget is not the best way,but it seems it's the only way to do that.I hope there's a APP to make it.
Thanks for your reply.

0 Karma

lcrielaa
Communicator

Hi Perlish,

There's several ways to do this but it depends on how you have access to this file:

  1. If you have direct access to the server, you could install a universal forwarder to monitor the file and forward it to your Splunk server.
  2. You could write a scripted input that would pickup the file (wget/ftp/scp?) and then load it into Splunk. Mind you that this wouldn't take care of duplicate entries so you'd have to arrange this sanity check yourself.
  3. If the server that has your file runs syslog (or rsyslog, or syslog-ng) you can possibly have the file sent to your Splunk server via syslog. This would require you enabling port 514 UDP/TCP (depending on the type of syslog) on your Splunk server.
  4. If the server that has the logfile can export the filesystem it has the logfile on, you can then mount the filesystem on your Splunk server (CIFS/SMB/NFS?) and ingest the file as if it was a local file. This will set the host variable to the local Splunk server so you might want to overwrite that with the remote server name.

As far as I know, there's no input for Splunk to monitor a file directly over HTTP from within Splunk itself.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...