Deployment Architecture

How to set the indexes for a single HTTP event collector input in an indexer cluster?

szabados
Communicator

I'm configuring what I believe is the first scenario here: http://dev.splunk.com/view/event-collector/SP-CAAAE73#scen1

I'm stuck at setting the indexes for this input. As I'm running an indexer cluster, my indexers are not visible on the forwarder, so I can't add the index I want to use to store the data coming from the HTTP Event Collector. I see only indexes which are defined on the forwarder itself, like main, summary, etc.

How can I set an index for this?
The other thing, the "Output Group" contains only the "None" value. However, I have outputs configured on this instance, since it is forwarding other events to my cluster. Will be these output settings applied to this by default, or do I have to configure them explicitly?

santiagn
Path Finder

any update on solution?

0 Karma

szabados
Communicator

yes: first I create the input on the web gui as described in the documentation, and then I go to the HFWD instance via the filesystem (rdp/ssh), /etc/apps/splunk_httpin (not 100% if this is the apps name, something similar)/local/inputs.conf

In this file, you change the index= to the index you want to use.

hope this helps

0 Karma

santiagn
Path Finder

thanks for the response, so for first step i leave outputgroup as none, second step of setup where it asks which index do you want to point to, what did you select? i still dont see my indexes, do i just leave it as main?

0 Karma

szabados
Communicator

Yes, I've left it as main, and then changed it to my "real" index in the *.conf file.

0 Karma

santiagn
Path Finder

thank you i will try this

0 Karma

gblock_splunk
Splunk Employee
Splunk Employee

If you have a single HEC instance forwarding to multiple indexers, then as Jeremiah said, you can create the indexes on the instance to allow you to select them via the Splunk UI. Or you can use our CLI / REST API which do not require the indexes be present. One caveat to this is if you specify the index in the payload (like using the "index" filed), those indexes must be present on the HEC instance. Regardless, the events will not be written locally to these indexes.

As far as forwarding, if you configure default output groups on the box, HEC will use them. The "Output Group" setting is to allow you to configure a group specifically for forwarding HEC events. It will not for example forward _internal or _introspection events.

0 Karma

Jeremiah
Motivator

If you would like to use the UI to set the index, you'll probably need to load an indexes.conf on the heavy forwarder that matches what you load on your indexers. Then, the forwarder will have the same list of indexes for you to choose from. Defining the indexes on the forwarder is fine, if you've configured your heavy forwarder correctly it will not write any data to the local indexes, just create their directory structure within the $SPLUNK_HOME/var/lib/splunk directory. Otherwise, you could define your settings directly in the inputs.conf file for the HEC and bypass the UI.

The forwarder will use your default output settings, even if the Output Group contains "None". You don't have to explicitly configure an additional output group unless you want the ability to route different data to different endpoints.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...