Getting Data In

How to load a comma separated file to Splunk by using REST API

igor1705
New Member

Is it possible to load a CSV file to Splunk only by using a REST API search?
I have Splunk installed on another machine and I need to have the ability to load data to the Splunk, let's say from any place all over the world.

Please advise how this REST API should look like if this possible.

Do I need to perform some additional changes in configuration?

Thanks.

0 Karma

jplumsdaine22
Influencer

I don't think you can use the OneShot input remotely. The file must be already available on your indexer. If you don't mind setting up copying the CSV across in a separate process (rsync for example, or even dropbox) , then you can start the oneshot index using the API (http://docs.splunk.com/Documentation/Splunk/6.3.3/RESTREF/RESTinput#data.2Finputs.2Foneshot)

Personally I think streaming the data using either a Universal Fowarder or a remote TCP input is a better idea. If your data is not suitable for streaming (ie it is not time series data) consider using a lookup or KV store instead of indexing the data (or use a database)

If you must send your CSV via HTTP then use the HTTP Event Collector (http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/UsetheHTTPEventCollector)

You could use a shellscript to step through the CSV and send line as individual events. For example (where foo bar and baz are your column headers) the following script will send an event for each line in your csv

while IFS="," read foo bar baz; 
    do curl -k  https://yourSplunk:8088/services/collector/event -H "Authorization: Splunk TOKEN" -d '{"event": "foo=$foo bar=$bar baz=$baz"}'; 
done < your.csv
0 Karma

igor1705
New Member

I tried to perform the following from another remote machine:

curl -k -u admin:admin -d "name=d:\splunk_test_data\load.csv" https://localhost:8089/services/data/inputs/monitor

"it was real host name instead from localhost"

I noticed that splunk adds this input only for local file, not remote, so it didn't work for adding files remotely.

Still looking way how to load files from remote host. Please advise.
Thanks

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...