Getting Data In

Universal Forwarder 6.4.0 to HEC

acamilo_2
New Member

Unfortunately, I have a few hundred hosts running 6.4 universal forwarder and I cannot upgrade them. I have a subset of hosts that need to send an application log to HEC on customer's splunk deployment but still send this and all the other logs to our splunk. We already have a few apps for the forwarder to handle different logs in a different way so I decided to create an app for those forwarders to send to the customer so that I can make sure that only the application log gets sent to them.
Hosts are Ubuntu 14.04
Splunk servers are on 16.04 and we run Splunk Enterprise 6.5.2
We don't use HEC in any of our hosts hence why I'm asking for help here.

I'm confident about all the config files except output.conf, I'm not clear on what defaultGroups does or if it's even necessary and the tcpout:app_upstart_logs stanza is also nebulous to me.

output.conf
[tcpout]
defaultGroup = ???
forwardedindex.filter.disable = true
indexAndForward = false
maxQueueSize = 250MB
maxConnectionsPerIndexer = 5

[tcpout:app_upstart_logs]
useACK = false
token = XXXXXXXXXXXXXXXXXXXXXXXXXX
server = customer.splunk-server.net:8080

This is my inputs.conf

[monitor:///var/log/upstart/]
disabled = false
sourcetype = app_upstart_logs
blacklist = (\.gz$|\.0$|.1$|\.2$|.3$|\.4$|.5$|\.6$|.7$|\.8$|\.9$|\.10$|\.report$|lost\+found)

PS: We are running some really old stuff. Feel free to roll your eyes like a teenage valley girl (is that still a thing?) I stand ashamed.

0 Karma

bandit
Motivator

An http event collector (hec) is essentially a heavy forwarder with the special hec listener. You would use the same sort of outputs.conf that you would use on any search head or heavy forwarder to route logs to your downstream indexers.

Example outputs.conf config:

# BASE SETTINGS

# TURN OFF INDEXING ON SEARCH HEAD OR HEAVY FORWARDER
[indexAndForward]
index = false

[tcpout]
# FORWARD ALL INTERNAL LOGS
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =

# round robin to all indexers
defaultGroup = all_indexers

[tcpout:all_indexers]
server = host500.acme.com:9997,host501.acme.com:9997,host502.acme.com:9997,host503.acme.com:9997,host504.acme.com:9997
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...