Getting Data In

Index all but one input

claudio_manig
Communicator

Guys-

I'm facing an (apparantely) challenging task:
I have a standalon splunk test instance which serves as a first point of ingestion for new inputs- however, what i want to achieve is the following:
As this instance runs on a windows server i want to have the configured os inputs (eventlog, perfmon) to be forwarded to our production instance and keep the rest local indexed.

What I have tried so far:

Easiest approach from my pov:
Created an outputs.conf

[tcpout:fwd_to_prod]
server = t800.skynet.net:9997

And a referring input in inputs.conf as follows:

[WinEventLog://Application]
_TCP_ROUTING = fwd_to_prod

Result: EVERYTHING is going to be forwarded to my production instance, including all internal stuff

After some research i tried it with the more complicated way, using a transforms to do so:
I had the same outputs.conf:

[tcpout:fwd_to_prod]
server = t800.skynet.net:9997

Added the following transforms stanza in transforms.conf:

#forward win events to prod splunk
[forward_prod]
DEST_KEY = _TCP_ROUTING
FORMAT = fwd_to_prod
REGEX = .

Then i referred to my transforms in my props.conf:

[WinEventLog://Application]
TRANSFORMS-App = forward_prod

Result: EVERYTHING is going to be forwarded to my production instance, including all internal stuff

Its weird somehow -
Yes i did read the docs, and yes, i'm aware that forwardandIndex and selectiveIndex is a way but it the docs all refer to the other way round, indexing one type and forward everything else! I don't want to set "_INDEX_AND_FORWARD_ROUTING" on all my inputs expect the windows one as you might understand.

I even tried to add a dummy tcp output group as default group in my outputs.conf with no effect - adding the "localhost" as the default target group resulted in no forwarding or indexing at all.

Any help is appreciated

Cheers

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Something like this should work

etc/system/local/outputs.conf (configure selective indexing)

[indexAndForward]
index=true
selectiveIndexing=true 

[tcpout:fwd_to_prod]
server = yourIndexer:9997

etc/system/local/inputs.conf (configure default behaviour as local indexing for all data inputs)

[default]
_INDEX_AND_FORWARD_ROUTING = MySecKey

Add the _TCP_ROUTING attribute to the stanzas of each input that you want to forward. E.g.

[WinEventLog://Application]
_TCP_ROUTING=fwd_to_prod

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

Something like this should work

etc/system/local/outputs.conf (configure selective indexing)

[indexAndForward]
index=true
selectiveIndexing=true 

[tcpout:fwd_to_prod]
server = yourIndexer:9997

etc/system/local/inputs.conf (configure default behaviour as local indexing for all data inputs)

[default]
_INDEX_AND_FORWARD_ROUTING = MySecKey

Add the _TCP_ROUTING attribute to the stanzas of each input that you want to forward. E.g.

[WinEventLog://Application]
_TCP_ROUTING=fwd_to_prod

claudio_manig
Communicator

Oh I see i missed the [default] option in inputs.conf to make it global - worked like a charm, thanks a lot!
Can you do me another favour and use my first hostname on your answer - thx

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...