Getting Data In

How to route same source to multiple indexers and their respective indexes ?

dm1
Contributor

I have two data sources (Syslog and Netflow) which I am collecting on a dedicated host, where I have installed a Universal Forwarder. It is acting as an intermediate forwarder. 

I have to route this data to Indexers of two different organisations on their respective indexes.

E.g

  1. OrgA
    1. Syslog needs to go to index=syslog_A
    2. Netflow needs to go to index=netflow_A
    3. Indexer is IndexerA:9997
  2. OrgB
    1. Same Syslog as above needs to go to index=syslog_B
    2. Same Netflow as above needs to go to index=netflow_B
    3. Indexer is IndexerB:9997
  3. MyOrg
    1. Only Splunk internal logs to IndexerMyOrg

Because this routing is based on metadata, I believe, I should be able to achieve this using universal forwarder.

Can someone please advise how I can achieve this ?

 

Tags (2)
0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @dm1 

_TCP_ROUTING setting in inputs conf works for your case. and you need to configure two tcpout indexer groups in outputs conf. Your config might look like as follows,

#inputs.conf

[monitor://<your_syslog_file_path>]
index=indexA
sourcetype=<syslog_st>
_TCP_ROUTING = indexerA-group

[monitor://<your_netflow_file_path>]
index=indexA
sourcetype=<netflow_st>
_TCP_ROUTING = indexerA-group 

[monitor://<your_syslog_file_path>]
index=indexB
sourcetype=<syslog_st>
_TCP_ROUTING = indexerB-group

[monitor://<your_netflow_file_path>]
index=indexB
sourcetype=<netflow_st>
_TCP_ROUTING = indexerB-group 

 

#outputs.conf

[tcpout:indexerA-group]
server=<indexerA-host>:9997

[tcpout:indexerB-group]
server=<indexerB-host>:9997

---

An upvote would  be appreciated and Accept solution if this reply helps!

 

0 Karma

dm1
Contributor

Hi @venkatasri , thanks for your reply.

 

but with the same monitor stanza, wouldn't Splunk just choose one setting and only forward to one indexer based on precedence ?

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@dm1 I too doubt about that as fishbucket ignores other monitors as duplicates just give a try! if not working then You might need HF to actually achieve in that case.

Can you follow this link - Solved: One source to two indexes - Splunk Community

---

An upvote would be appreciated if this reply helps!

0 Karma

dm1
Contributor

From this link - https://docs.splunk.com/Documentation/Splunk/8.2.1/Forwarding/Routeandfilterdatad#Route_inputs_to_sp... it seems possible to route to two different indexers, but my only main challenge is assigning two indexes to same source

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@dm1  That's right indexers is not a problem can be done in UF. 

indexes setting you need HF help.

0 Karma
Get Updates on the Splunk Community!

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...

IM Landing Page Filter - Now Available

We’ve added the capability for you to filter across the summary details on the main Infrastructure Monitoring ...

Dynamic Links from Alerts to IM Navigators - New in Observability Cloud

Splunk continues to improve the troubleshooting experience in Observability Cloud with this latest enhancement ...