Hello,
As the question states, i'm looking to send events from a universal forwarder to a heavy forwarder to have filtered. Once filtered, i'd like to change the sourcetype. I have not implemented this yet. This is for me to propose to upper management to agree on. I want to make sure the props/transforms piece is correct. I think the filtering is good, however i just want to make sure the syntax is all good.
I've listed my config and config details:
ON UNIVERSAL FORWARDER
[monitor://c:\program files\app1\web.log]
_TCP_ROUTING = filter_heavy_forwarders
index = cmis_index
ON UNIVERSAL FORWARDER
[tcpout]
defaultGroup=infosec_indexers
[tcpout:infosec_indexers]
autoLB = true
server = infosec_server1:9997,infosec_server2:9997,infosec_server3:9997…,infosec_server16:9997
[tcpout:cmis_indexers]
autoLB = true
server = cmis_server1:9997
[tcpout:filter_heavy_forwarders]
autoLB = true
ON HEAVY FORWARDER
[app1_web_logs]
TRANSFORMS-routing = app1_web_filter
ON HEAVY FORWARDER
[app1_web_filter]
REGEX = (Events|To|Filter)
DEST_KEY = _TCP_ROUTING
FORMAT = cmis_indexers
[app1_cmis_web_st]
DEST_KEY = MetaData:Sourcetype
ON HEAVY FORWARDER
[tcpout]
defaultGroup=none
[tcpout:cmis_indexers]
autoLB = true
Hi johnmvang,
only just a few information:
in UFs outputs.conf I don't see
[tcpout-server://infosec_server1:9997]
[tcpout-server://infosec_server2:9997]
[tcpout-server://infosec_server3:9997]
[tcpout-server://infosec_server16:9997]
but probably you missed these rows only in the question.
On HF, you send all transforming logs only to cmis_indexers?
if yes you don't need in props.conf TRANSFORMS-routing = app1_web_filter
and the relative stanza in transforms.conf.
In addition I suggest to perform selective addressing directly in UFs.
Anyway, I think that the problem is in HFs transforms.conf: the REGEX row is missing, so add REGEX = .
to the [app1_cmis_web_st] stanza.
Bye.
Giuseppe
You can use btool to validate your syntax.
I notice you don't mention both tcpout's within the outputs.conf but this might be from the universal forwarder only.
i updated my question with the unifwd outputs. But let me look into the btool and i'll come back.
Thanks,
John