Splunk Dev

Splunk using HTTP Event Collector (HEC) with services/collector/raw endpoint to send whole datafile

RetailOperation
Explorer

Hello,

we want to send a whole file via the http event collector to the endpoint services/collector/raw as descirbed here:
http://dev.splunk.com/view/event-collector/SP-CAAAE8Y

"curl https://http-inputs-.splunkcloud.com/services/collector/raw -H "X-Splunk-Request-Channel: FE0ECFAD-13D5-401B-847D-77833BD77131" -H "Authorization: Splunk BD274822-96AA-4DA6-90EC-18940FB2414C" -d '' -v"

As we are on Splunk 7 we do not use the "X-Splunk-Request-Channel"

The data is recieved by an heavy forwarder and parsed to the indexer cluster.
The data is arriving at the assigned index but the problem is that splunk handles the whole file as one event.
I created a sourcetype on a test environment which breaks up the events in single events when i upload the file manually.
But when i send this file to our productive environment via the HEC with /raw endpoint the file is still indexed as one event.

I tried to configure the sourcetype on all indexers in the index cluster -> was nor working
I tried to configure the sourcetype also at the heavy forwarders -> not working

Any suggestions what to do here?

thanks in advance

Tags (1)
0 Karma

starcher
Influencer

The event endpoint expects properly formed JSON and things broken into events. It bypasses parsing queues. If you send to RAW you treat it like any file pickup ingestion and do your own line breaking etc as you would any text stream multi line event.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...