Hi All,
Facing few challlenges, mine is playing around with the same transforms.
I'm trying to achieve the same source data to forward to two different logical indexes and two different indexers groups.
Below is my senrio.
In props.conf used
[source::Dual_Data_Testing]
TRANSFORMS-source = Stan1, Stan2
In transforms.conf
[Stan1]
SOURCE_KEY = MetaData:Source
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = Index1
DEST_KEY = _TCP_ROUTING
FORMAT = IndexerGroup1
[Stan2]
SOURCE_KEY = MetaData:Source
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = Index2
DEST_KEY = _TCP_ROUTING
FORMAT = IndexerGroup2
Currently the above conf is not working.
Please any suggestion can we workaround for this ?
Thanks,
Arun Sunny
try this
inputs.conf
[monitor://filepath1]
index=index1
_TCP_ROUTING = indexergroup1
[monitor://filepath1]
index=index2
_TCP_ROUTING = indexergroup2
Outputs.conf
[tcpout:indexergroup1]
server=server1:9997
[tcpout:indexergroup2]
server=server2:9997
Actually, I was trying for one of the DB input sources, so I cant duplicate the monitor stanza in inputs.conf
Thanks,
And I believe we can play around only once in _MetaData key values in transforms.conf .
yeah thus why i have two different sourcetype for a source. But you mentioned that it is writing to only one sourcetype. May be you can try one with _TCP_ROUTING and another with _SYSLOG_ROUTING.
Check the below link,
http://docs.splunk.com/Documentation/Splunk/6.6.2/Forwarding/Routeandfilterdatad
Topic: Replicate a subset of data to a third-party system
try this,
#props.conf
[source::Dual_Data_Testing]
sourcetype=sourcetype1
[source::Dual_Data_Testing]
sourcetype=sourcetype2
[sourcetype1]
TRANSFORMS-index_outputgroup1 = overrideindex1,outputgroup1
[sourcetype2]
TRANSFORMS-index_outputgroup2 = overrideindex2,outputgroup2
Transforms.conf
[overrideindex1]
DEST_KEY =_MetaData:Index
REGEX = .
FORMAT = my_new_index1
[overrideindex2]
DEST_KEY =_MetaData:Index
REGEX = .
FORMAT = my_new_index2
[outputgroup1]
REGEX=(.)
DEST_KEY=_TCP_ROUTING
FORMAT=outputgroup11
[outputgroup2]
REGEX=(.)
DEST_KEY=_TCP_ROUTING
FORMAT=outputgroup22
Outputs.conf
[tcpout:outputgroup11]
server=server1:9997
[tcpout:outputgroup22]
server=server1:9997
Its working fine for one output group and other is completely stopped sending events 😞 .
Did you check data is writing on both the index and sourcetype.
Yes, I checked. It's writing only to the first index and passing the same to group1 indexers.
Hi arunsunny,
do you want to send all logs to:
if the first, you don't need to configure props and transforms, you have only to configure outputs.conf
[tcpout:Group1]
defaultGroup = default-autolb-group
[tcpout-server://xx.xxx.xxx.xx:9997]
[tcpout-server://yy.yyy.yyy.yy:9997]
[tcpout:default-autolb-group]
server = xx.xxx.xxx.xx:9997, yy.yyy.yyy.yy:9997
disabled = false
[tcpout:Group2]
server=aa.aaa.aaa.aa:9997, bb.bbb.bbb.bb:9997
disabled = false
[tcpout-server://aa.aaa.aaa.aa:9997]
[tcpout-server://bb.bbb.bbb.bb:9997]
If the second, follow http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad
In other words you have to configure an outputs.conf as above and in every inputs.conf stanza put:
Bye.
Giuseppe