Getting Data In

Index all but one input

claudio_manig
Communicator

Guys-

I'm facing an (apparantely) challenging task:
I have a standalon splunk test instance which serves as a first point of ingestion for new inputs- however, what i want to achieve is the following:
As this instance runs on a windows server i want to have the configured os inputs (eventlog, perfmon) to be forwarded to our production instance and keep the rest local indexed.

What I have tried so far:

Easiest approach from my pov:
Created an outputs.conf

[tcpout:fwd_to_prod]
server = t800.skynet.net:9997

And a referring input in inputs.conf as follows:

[WinEventLog://Application]
_TCP_ROUTING = fwd_to_prod

Result: EVERYTHING is going to be forwarded to my production instance, including all internal stuff

After some research i tried it with the more complicated way, using a transforms to do so:
I had the same outputs.conf:

[tcpout:fwd_to_prod]
server = t800.skynet.net:9997

Added the following transforms stanza in transforms.conf:

#forward win events to prod splunk
[forward_prod]
DEST_KEY = _TCP_ROUTING
FORMAT = fwd_to_prod
REGEX = .

Then i referred to my transforms in my props.conf:

[WinEventLog://Application]
TRANSFORMS-App = forward_prod

Result: EVERYTHING is going to be forwarded to my production instance, including all internal stuff

Its weird somehow -
Yes i did read the docs, and yes, i'm aware that forwardandIndex and selectiveIndex is a way but it the docs all refer to the other way round, indexing one type and forward everything else! I don't want to set "_INDEX_AND_FORWARD_ROUTING" on all my inputs expect the windows one as you might understand.

I even tried to add a dummy tcp output group as default group in my outputs.conf with no effect - adding the "localhost" as the default target group resulted in no forwarding or indexing at all.

Any help is appreciated

Cheers

0 Karma
1 Solution

somesoni2
Revered Legend

Something like this should work

etc/system/local/outputs.conf (configure selective indexing)

[indexAndForward]
index=true
selectiveIndexing=true 

[tcpout:fwd_to_prod]
server = yourIndexer:9997

etc/system/local/inputs.conf (configure default behaviour as local indexing for all data inputs)

[default]
_INDEX_AND_FORWARD_ROUTING = MySecKey

Add the _TCP_ROUTING attribute to the stanzas of each input that you want to forward. E.g.

[WinEventLog://Application]
_TCP_ROUTING=fwd_to_prod

View solution in original post

somesoni2
Revered Legend

Something like this should work

etc/system/local/outputs.conf (configure selective indexing)

[indexAndForward]
index=true
selectiveIndexing=true 

[tcpout:fwd_to_prod]
server = yourIndexer:9997

etc/system/local/inputs.conf (configure default behaviour as local indexing for all data inputs)

[default]
_INDEX_AND_FORWARD_ROUTING = MySecKey

Add the _TCP_ROUTING attribute to the stanzas of each input that you want to forward. E.g.

[WinEventLog://Application]
_TCP_ROUTING=fwd_to_prod

claudio_manig
Communicator

Oh I see i missed the [default] option in inputs.conf to make it global - worked like a charm, thanks a lot!
Can you do me another favour and use my first hostname on your answer - thx

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...