Getting Data In

What is the best and fastest way to transport .log files by syslog?

yangban
Explorer

Hi we are trying to transport several .log files to a Forwarder by syslog.

We used some bash scripts to do so, but it was just way too slow for us.(nearly 1MB/min)

Is there anyone know a faster way?

Thanks.

0 Karma
1 Solution

FrankVl
Ultra Champion

You might want to take inspiration from this .conf17 talk: https://conf.splunk.com/conf-online.html?search=FN122172#/

Which uses a simple python script to send data to a HEC receiver.

Alternatively:
install a UF on the system where the log files are.
Or write a faster bash script (perhaps you can share what you had now).
Or configure an rsyslog/syslog-ng daemon to read from those files and then forward to a syslog receiver on the Forwarder.

View solution in original post

0 Karma

FrankVl
Ultra Champion

You might want to take inspiration from this .conf17 talk: https://conf.splunk.com/conf-online.html?search=FN122172#/

Which uses a simple python script to send data to a HEC receiver.

Alternatively:
install a UF on the system where the log files are.
Or write a faster bash script (perhaps you can share what you had now).
Or configure an rsyslog/syslog-ng daemon to read from those files and then forward to a syslog receiver on the Forwarder.

0 Karma

yangban
Explorer

Thanks! I might try HEC!

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...