Getting Data In

Forwarding select data in my environment.

Steve_Litras
Path Finder

I'm trying to take data from specific systems and, after indexing it, forward it to a third party for other analysis. I have data coming from light forwarders on the systems in question, and going through a forwarding layer before it gets to the indexing hosts. I applied, basically, the recipe as defined in the documentation on the indexers:

props.conf:

[host::*WCA*]
TRANSFORMS-routing = send_to_syslog

transforms.conf (note, splunk complains about not having a REGEX entry - I've tried it with .* as the REGEX to no avail):

[send_to_syslog]
DEST_KEY = _SYSLOG_ROUTING
FORMAT = SW_syslog_group

outputs.conf:

[syslog:SW_syslog_group]
disabled=false
server = 10.20.30.40:514

I'm wondering if I need to apply the props.conf and the transforms.conf at the forwarder layer, and if I do that, will it still properly index, or will it forward out prior to indexing?

thanks Steve

Tags (1)
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

So you have LWF->Intermediate Hvy Forwarder->Indexer? If so, the problem is that the TRANSFORMS/parsing phase will not occur on the indexer. However, you're right that you don't want to set this on the intermediate forwarder layer either, because I'm pretty sure it won't carry through to the indexer. I would try it though, because if it does work it will be a lot easier.

Otherwise, I'd would instead set up an extra Splunk TCP port on the indexer:

# regular tcp port
[splunktcp://9997]

# special tcp port everything sent here goes to syslog out
[splunktcp://9999]
_SYSLOG_ROUTING = SW_syslog_group

Then on the intermediate layer, you route to the appropriate Splunk TCP output group:

[send_to_syslog_port]
DEST_KEY = _TCP_ROUTING
FORMAT = SW_splunk_syslog_port_group

where the SW_splunk_syslog_port_group goes to port 9999. (The default group is the one that goes to 9997.)


It's also possible to force the indexer to run parsed data (from the Hvy Forwarder) through the parse queue again. This has problems, depending on your sourcetype rules, because some transforms might get run more than once, which means you'll need to go through configs on the indexers and disable default or previously inactive configs to prevent that. Once you've done that, you can add into a global inputs.conf:

[splunktcp]
route=has_key:_utf8:parsingQueue;has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue

This overrides the default rule that only parses data from a LWF, and parses everything from any forwarder. I'm not sure if you could do:

[splunktcp://9999]
route=has_key:_utf8:parsingQueue;has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue

and only apply it to data on one port. If it does work, and if you need or want to distinguish it, that might not be a bad idea. This isn't great, because you're put

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

So you have LWF->Intermediate Hvy Forwarder->Indexer? If so, the problem is that the TRANSFORMS/parsing phase will not occur on the indexer. However, you're right that you don't want to set this on the intermediate forwarder layer either, because I'm pretty sure it won't carry through to the indexer. I would try it though, because if it does work it will be a lot easier.

Otherwise, I'd would instead set up an extra Splunk TCP port on the indexer:

# regular tcp port
[splunktcp://9997]

# special tcp port everything sent here goes to syslog out
[splunktcp://9999]
_SYSLOG_ROUTING = SW_syslog_group

Then on the intermediate layer, you route to the appropriate Splunk TCP output group:

[send_to_syslog_port]
DEST_KEY = _TCP_ROUTING
FORMAT = SW_splunk_syslog_port_group

where the SW_splunk_syslog_port_group goes to port 9999. (The default group is the one that goes to 9997.)


It's also possible to force the indexer to run parsed data (from the Hvy Forwarder) through the parse queue again. This has problems, depending on your sourcetype rules, because some transforms might get run more than once, which means you'll need to go through configs on the indexers and disable default or previously inactive configs to prevent that. Once you've done that, you can add into a global inputs.conf:

[splunktcp]
route=has_key:_utf8:parsingQueue;has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue

This overrides the default rule that only parses data from a LWF, and parses everything from any forwarder. I'm not sure if you could do:

[splunktcp://9999]
route=has_key:_utf8:parsingQueue;has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue

and only apply it to data on one port. If it does work, and if you need or want to distinguish it, that might not be a bad idea. This isn't great, because you're put

Steve_Litras
Path Finder

Ok - so I finally was able to get this in place (change windows and firewalls oh my), and it works, but the data being sent to the 3rd party no longer has the original host header in it. Is that due to truncation, or is there something I can do to specifically make sure that data goes with the data?

0 Karma

Steve_Litras
Path Finder

If I set up the alternate port, as you suggested, would the data still be indexed? Do I, in that case need to specify:

indexAndForward = true

for the "special" splunktcp port?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...