Getting Data In

Index all but one input

claudio_manig
Communicator

Guys-

I'm facing an (apparantely) challenging task:
I have a standalon splunk test instance which serves as a first point of ingestion for new inputs- however, what i want to achieve is the following:
As this instance runs on a windows server i want to have the configured os inputs (eventlog, perfmon) to be forwarded to our production instance and keep the rest local indexed.

What I have tried so far:

Easiest approach from my pov:
Created an outputs.conf

[tcpout:fwd_to_prod]
server = t800.skynet.net:9997

And a referring input in inputs.conf as follows:

[WinEventLog://Application]
_TCP_ROUTING = fwd_to_prod

Result: EVERYTHING is going to be forwarded to my production instance, including all internal stuff

After some research i tried it with the more complicated way, using a transforms to do so:
I had the same outputs.conf:

[tcpout:fwd_to_prod]
server = t800.skynet.net:9997

Added the following transforms stanza in transforms.conf:

#forward win events to prod splunk
[forward_prod]
DEST_KEY = _TCP_ROUTING
FORMAT = fwd_to_prod
REGEX = .

Then i referred to my transforms in my props.conf:

[WinEventLog://Application]
TRANSFORMS-App = forward_prod

Result: EVERYTHING is going to be forwarded to my production instance, including all internal stuff

Its weird somehow -
Yes i did read the docs, and yes, i'm aware that forwardandIndex and selectiveIndex is a way but it the docs all refer to the other way round, indexing one type and forward everything else! I don't want to set "_INDEX_AND_FORWARD_ROUTING" on all my inputs expect the windows one as you might understand.

I even tried to add a dummy tcp output group as default group in my outputs.conf with no effect - adding the "localhost" as the default target group resulted in no forwarding or indexing at all.

Any help is appreciated

Cheers

0 Karma
1 Solution

somesoni2
Revered Legend

Something like this should work

etc/system/local/outputs.conf (configure selective indexing)

[indexAndForward]
index=true
selectiveIndexing=true 

[tcpout:fwd_to_prod]
server = yourIndexer:9997

etc/system/local/inputs.conf (configure default behaviour as local indexing for all data inputs)

[default]
_INDEX_AND_FORWARD_ROUTING = MySecKey

Add the _TCP_ROUTING attribute to the stanzas of each input that you want to forward. E.g.

[WinEventLog://Application]
_TCP_ROUTING=fwd_to_prod

View solution in original post

somesoni2
Revered Legend

Something like this should work

etc/system/local/outputs.conf (configure selective indexing)

[indexAndForward]
index=true
selectiveIndexing=true 

[tcpout:fwd_to_prod]
server = yourIndexer:9997

etc/system/local/inputs.conf (configure default behaviour as local indexing for all data inputs)

[default]
_INDEX_AND_FORWARD_ROUTING = MySecKey

Add the _TCP_ROUTING attribute to the stanzas of each input that you want to forward. E.g.

[WinEventLog://Application]
_TCP_ROUTING=fwd_to_prod

claudio_manig
Communicator

Oh I see i missed the [default] option in inputs.conf to make it global - worked like a charm, thanks a lot!
Can you do me another favour and use my first hostname on your answer - thx

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...