Getting Data In

Splunk HEC Integration. Where to enable? Deployment Server or Log Server?

lloydknight
Builder

Hello,

Here's our Splunk setup:

3 Indexers (not clustered)
1 Search Head/Deployment Server
1 Log Server (acts like Heavy Forwarder)

All Splunk instances are reporting to Deployment Server and indexes are app-based.
Log Server is basically a log server where other integrations that doesn't require Forwarder use this to dump their logs and we push an application from Deployment server to monitor the logs in the Log server.

Questions:
1. Where should I enable HEC?
This will be my first time using an HEC and I'm not sure if I'll enable it to Log server or Deployment Server.

2. If I will enable it to our Log Server, will there be API logs that will be dumped/generated somewhere that I can just monitor using deployed monitor apps from Deployment Server?

3. If I will enable it to our Deployment Server, my concern here is if it will generate actual logs which should be done in Log Server and if it will not generate logs on the server, what is the behavior of the API logs upon enabling it to Deployment server?

4. Will the API logs directly be indexed on the defined index? (i.e. index=api_logs)

5. High level explanation for Indexer Acknowledgement?

Can someone kindly answer all my 5 questions?

PS.
Already read this btw:
http://dev.splunk.com/view/event-collector/SP-CAAAE73

Much appreciated!

0 Karma
1 Solution

skalliger
SplunkTrust
SplunkTrust

Hi,

you might want to read this about HEC: https://www.splunk.com/blog/2017/03/30/syslog-ng-and-hec-scalable-aggregated-data-collection-in-splu...

Should answer 1-4 (1. syslog, 2. I don't get your question, but the blog should clear that, 3. In general, you want to forward your splunk data to your indexers anyways, see link below, 4. yes).

Indexer acknowledgment works like this: a fowarder sends data to the indexer. An indexer has to acknowledge the data before the forwarder sends other data. In case of no ACK, the data will be re-sent. An indexer acknowledges data after it has been written to the disk.

Additional links:
https://docs.splunk.com/Documentation/Splunk/6.6.0/DistSearch/Forwardsearchheaddata
https://docs.splunk.com/Documentation/Splunk/6.6.0/Forwarding/Protectagainstlossofin-flightdata

Skalli

View solution in original post

0 Karma

skalliger
SplunkTrust
SplunkTrust

Hi,

you might want to read this about HEC: https://www.splunk.com/blog/2017/03/30/syslog-ng-and-hec-scalable-aggregated-data-collection-in-splu...

Should answer 1-4 (1. syslog, 2. I don't get your question, but the blog should clear that, 3. In general, you want to forward your splunk data to your indexers anyways, see link below, 4. yes).

Indexer acknowledgment works like this: a fowarder sends data to the indexer. An indexer has to acknowledge the data before the forwarder sends other data. In case of no ACK, the data will be re-sent. An indexer acknowledges data after it has been written to the disk.

Additional links:
https://docs.splunk.com/Documentation/Splunk/6.6.0/DistSearch/Forwardsearchheaddata
https://docs.splunk.com/Documentation/Splunk/6.6.0/Forwarding/Protectagainstlossofin-flightdata

Skalli

0 Karma

lloydknight
Builder

big help. thanks

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...