Getting Data In

How to route same source to multiple indexers and their respective indexes ?

dm1
Contributor

I have two data sources (Syslog and Netflow) which I am collecting on a dedicated host, where I have installed a Universal Forwarder. It is acting as an intermediate forwarder. 

I have to route this data to Indexers of two different organisations on their respective indexes.

E.g

  1. OrgA
    1. Syslog needs to go to index=syslog_A
    2. Netflow needs to go to index=netflow_A
    3. Indexer is IndexerA:9997
  2. OrgB
    1. Same Syslog as above needs to go to index=syslog_B
    2. Same Netflow as above needs to go to index=netflow_B
    3. Indexer is IndexerB:9997
  3. MyOrg
    1. Only Splunk internal logs to IndexerMyOrg

Because this routing is based on metadata, I believe, I should be able to achieve this using universal forwarder.

Can someone please advise how I can achieve this ?

 

Tags (2)
0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @dm1 

_TCP_ROUTING setting in inputs conf works for your case. and you need to configure two tcpout indexer groups in outputs conf. Your config might look like as follows,

#inputs.conf

[monitor://<your_syslog_file_path>]
index=indexA
sourcetype=<syslog_st>
_TCP_ROUTING = indexerA-group

[monitor://<your_netflow_file_path>]
index=indexA
sourcetype=<netflow_st>
_TCP_ROUTING = indexerA-group 

[monitor://<your_syslog_file_path>]
index=indexB
sourcetype=<syslog_st>
_TCP_ROUTING = indexerB-group

[monitor://<your_netflow_file_path>]
index=indexB
sourcetype=<netflow_st>
_TCP_ROUTING = indexerB-group 

 

#outputs.conf

[tcpout:indexerA-group]
server=<indexerA-host>:9997

[tcpout:indexerB-group]
server=<indexerB-host>:9997

---

An upvote would  be appreciated and Accept solution if this reply helps!

 

0 Karma

dm1
Contributor

Hi @venkatasri , thanks for your reply.

 

but with the same monitor stanza, wouldn't Splunk just choose one setting and only forward to one indexer based on precedence ?

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@dm1 I too doubt about that as fishbucket ignores other monitors as duplicates just give a try! if not working then You might need HF to actually achieve in that case.

Can you follow this link - Solved: One source to two indexes - Splunk Community

---

An upvote would be appreciated if this reply helps!

0 Karma

dm1
Contributor

From this link - https://docs.splunk.com/Documentation/Splunk/8.2.1/Forwarding/Routeandfilterdatad#Route_inputs_to_sp... it seems possible to route to two different indexers, but my only main challenge is assigning two indexes to same source

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@dm1  That's right indexers is not a problem can be done in UF. 

indexes setting you need HF help.

0 Karma
Get Updates on the Splunk Community!

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...