Getting Data In

Send data via Splunk REST API to a Heavy Forwarder requires index on the HF?

xiaoweiwu
New Member

We have indexer cluster setup. We are trying to use the REST API on the Heavy Forwarder to receive data update. But i am not sure if creating index on the HF is required to make sure indexer gets the data.

I do see an earlier posting is similar to this one. Since it's an old posting, i just want to confirm If using REST API on the HF, while output.conf can point to the indexer/cluster would work, with no index stored on the HF.
https://answers.splunk.com/answers/38176/data-indexed-via-rest-api-then-forwarded-on-to-another-inde...

0 Karma

koshyk
Super Champion
  • indexes.conf is NOT required at HF.
  • But it is very important that your HF forwards (outputs.conf) to Indexers
  • props.conf/tranforms.conf is required at HF and indextime changes happen at HF though it sends data to indexers
0 Karma

xiaoweiwu
New Member

I think i've gone on a tangent. All I wanted to know is for REST API, it's pushing data to this URI.
http://[Splunk IP]:8089/services/receivers/simple?index=myindex&source=sccm.py&sourcetype=web_event
If no index is required on the HF, then it will be sent to the indexer directly.

Is that correct?

0 Karma

xiaoweiwu
New Member

i do want to mention. I have an index on the HF. I am not sure why it is not in the indexes.conf.

0 Karma

koshyk
Super Champion

it might be one or another app contain indexes.conf
if you do a find on all apps with indexes.conf or a btool then it shows which app have created it incorrectly

/opt/splunk/bin/splunk cmd btool indexes list --debug > /tmp/indexes.btool

in your HF

0 Karma

xiaoweiwu
New Member

ok, I was able to find on the HF at this location for the indexes.conf.

/opt/splunk/etc/apps/search/local

And I've found the index in that indexes.conf file.
[myindex]
coldPath = $SPLUNK_DB/myindex/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB/myindex/db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB/myindex/thaweddb

0 Karma

xiaoweiwu
New Member

I did look at my HF.
There is no entry in the indexes.conf at this location /opt/splunk/etc/system/default.

Here is outputs.conf in /opt/splunk/etc/system/local

[tcpout]
defaultGroup = default-autolb-group
indexAndForward = 1

[tcpout:default-autolb-group]
server = xx.xx.xx.xx:9997

[tcpout-server://xx.xx.xx.xx:9997]

0 Karma

starcher
SplunkTrust
SplunkTrust

HTTP Event collector is better choice than trying to post to the api.

0 Karma

xiaoweiwu
New Member

Not to ignore your comments. Why is the event collector is better? Just want to know your view point.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...