Dashboards & Visualizations

It is possible to "tag" all data coming into a particular HEC token?

twinspop
Influencer

I have about 50 different tokens. I want data from one particular token to get some metadata added to it. Unfortunately, it doesn't appear that the _meta directive works for http in inputs.conf. Is it possible to replicate this functionality some how?

0 Karma

ameizeraitis
New Member

You can use method i have implemented with DS distributed bash script automation, which does following with every single HEC input on each server in hfw pool:
First, append existing http stanzas in inputs.conf with "fake" output group, like

[http://hec_input_1]

outputgroup = out01

Define those fake outputs in outputs conf like this:

[tcpgroup:out01]

server=127.0.0.1:9001

Now we need to set some listener on internal loop input dedicated port that "tags" the data:

[splunktcp://9001]

_meta = HecName::192.168.0.1:hec_input_1

Repeat all this for for all your hec inputs, make each of it have it's own outputgroup and tcpsplunk port listener, restart splunk and enjoy:

|tstats count where index=hec_index by HecName

0 Karma

MuS
SplunkTrust
SplunkTrust

The inputs name will translate into a source::http:InputNameHere which in turn should be useable in props.conf
But I must admit, I have not yet tried it 😉

cheers, MuS

twinspop
Influencer

Well shoot. If the sending application sets source, that overrides the default above, which means the transform doesn't fire. So still back to the old problem: How to guarantee a transform gets applied to every single event that came through a particular token's input def?

0 Karma

MuS
SplunkTrust
SplunkTrust

In this case, did someone say cough cribl cough 😉

twinspop
Influencer

We're testing it, but not ready to roll into production. Yet. 🙂 Very promising!

0 Karma

twinspop
Influencer

Perfect! I had no idea that was a thing. I feel like I gained a new superpower.

0 Karma

MuS
SplunkTrust
SplunkTrust

Glad I could help - Enjoy the new superpower 🙂

cheers, MuS

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi twinspop,

you can always use the good old props.conf / transforms.conf approach and add a meta field this way. Here is an example transforms.conf I use to add the hostname of the parsing HWF to events:

[add-relay-info-to-meta]
FORMAT = splunk_hwf::HostNameHere
REGEX = .
WRITE_META = true

Yes, it is a static value but I assume you will not change your HEC input too often 😉

Hope this helps ...

cheers, MuS

0 Karma

twinspop
Influencer

Yeah, a transform is where i was headed, but I don't see any foolproof way to identify only those those logs, and ALL those logs, that originate on 1 particular token. The token value and the input name are not things I can key off of in props as far as i know.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...