Getting Data In

Send data via Splunk REST API to a Heavy Forwarder requires index on the HF?

xiaoweiwu
New Member

We have indexer cluster setup. We are trying to use the REST API on the Heavy Forwarder to receive data update. But i am not sure if creating index on the HF is required to make sure indexer gets the data.

I do see an earlier posting is similar to this one. Since it's an old posting, i just want to confirm If using REST API on the HF, while output.conf can point to the indexer/cluster would work, with no index stored on the HF.
https://answers.splunk.com/answers/38176/data-indexed-via-rest-api-then-forwarded-on-to-another-inde...

0 Karma

koshyk
Super Champion
  • indexes.conf is NOT required at HF.
  • But it is very important that your HF forwards (outputs.conf) to Indexers
  • props.conf/tranforms.conf is required at HF and indextime changes happen at HF though it sends data to indexers
0 Karma

xiaoweiwu
New Member

I think i've gone on a tangent. All I wanted to know is for REST API, it's pushing data to this URI.
http://[Splunk IP]:8089/services/receivers/simple?index=myindex&source=sccm.py&sourcetype=web_event
If no index is required on the HF, then it will be sent to the indexer directly.

Is that correct?

0 Karma

xiaoweiwu
New Member

i do want to mention. I have an index on the HF. I am not sure why it is not in the indexes.conf.

0 Karma

koshyk
Super Champion

it might be one or another app contain indexes.conf
if you do a find on all apps with indexes.conf or a btool then it shows which app have created it incorrectly

/opt/splunk/bin/splunk cmd btool indexes list --debug > /tmp/indexes.btool

in your HF

0 Karma

xiaoweiwu
New Member

ok, I was able to find on the HF at this location for the indexes.conf.

/opt/splunk/etc/apps/search/local

And I've found the index in that indexes.conf file.
[myindex]
coldPath = $SPLUNK_DB/myindex/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB/myindex/db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB/myindex/thaweddb

0 Karma

xiaoweiwu
New Member

I did look at my HF.
There is no entry in the indexes.conf at this location /opt/splunk/etc/system/default.

Here is outputs.conf in /opt/splunk/etc/system/local

[tcpout]
defaultGroup = default-autolb-group
indexAndForward = 1

[tcpout:default-autolb-group]
server = xx.xx.xx.xx:9997

[tcpout-server://xx.xx.xx.xx:9997]

0 Karma

starcher
Influencer

HTTP Event collector is better choice than trying to post to the api.

0 Karma

xiaoweiwu
New Member

Not to ignore your comments. Why is the event collector is better? Just want to know your view point.

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...