Monitoring Splunk

enabling HTTP Event Collector for docker container on splunk clustered setup

sim_tcr
Communicator

We tested HTTP Event Collector for a docker container by starting the container with below,

--log-driver=splunk --log-opt splunk-token=<token> --log-opt splunk-url=https://<splunkserver>:8088 --log-opt splunk-insecureskipverify=true

Now we want to send the events to our actual splunk setup. We are on search head clustering and index clustering enabled, with a separate shc deployer, deployment server and index master servers.

  • Which server we should enable the HTTP Event Collector?
  • How can i specify the index where the events should be forwarded?

Thanks,

Tags (1)
0 Karma

outcoldman
Communicator

Hi @sim_tcr,

If you are just integrating Docker with Splunk, have you seen our solution for collecting logs and metrics https://splunkbase.splunk.com/app/3723/? You can find some documentation https://www.outcoldsolutions.com/docs/ demos and screencasts. We have also comparison table with official Splunk Logging Driver https://www.outcoldsolutions.com/docs/collectorfordocker/#comparing-with-splunk-logging-driver (btw, I am the original author of this driver).

0 Karma

mattymo
Splunk Employee
Splunk Employee

Hi @sim_tcr!

The best advice for you would require a peek at your existing architecture and data volumes, but you should start by having a look at dev.splunk.com for HEC architectures:

http://dev.splunk.com/view/event-collector/SP-CAAAE73

If you are just getting started you could look at having a Heavy forwarder in place to catch the HEC traffic and pass it to your indexers, or a pool of HFs, or you could also just enable it on all your indexers - although you will want to be careful with that depending on the volume of traffic you expect.

As for routing to an index, check out the docs on HEC tokens. You can set the index/sourcetype there!

http://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector

- MattyMo

sim_tcr
Communicator

Thank you for responding.
I think we want to go with http://dev.splunk.com/view/event-collector/SP-CAAAE73#scen1
We are at 6.3.3 and referring http://docs.splunk.com/Documentation/Splunk/6.3.3/Data/UsetheHTTPEventCollector
when checking "select allowed indexes" field, we do not see all our indexes from the index cluster listed there.
Do you know what are we missing?

0 Karma

sloshburch
Splunk Employee
Splunk Employee

Great answer @mmodestino!

@sim_tcr - Remember that if you start with Scenario 1 and you have to change/rebuilt the HEC server, you'll need to update all the clients (apps) sending data. As such, use a VIP or a Load Balancer even if pointing to that one instance. This will increase availability and failover options besides allowing for scalability later and essentially making the transition to Scenario 3 seamless.

0 Karma

mattymo
Splunk Employee
Splunk Employee

You need to tell your HF about them in indexes.conf

Grab the TA from the indexer cluster and use that indexes.conf on the HF.

- MattyMo
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...