Heavy Forwarder between Exchange servers and Indexers

[ddeighton]

When we first installed the Exchange app, we configured the Universal Forwarders to send our logs directly to the Indexers and all was good. We made a change to the Universal Forwarders on the Exchange servers so that the logs now go through our Heavy Forwarder. At that point, our User Count dropped to 25 users (which is way below the actual number of users). After doing some investigation, we realized that "msexchange:2010:mailbox-usage" log lines were merged. This is what caused the problem with the User count (and possibly caused other issues).

How do we correct the line merging problem?

[ddeighton]

By default, SHOULD_LINEMERGE is set to true in /opt/splunk/etc/system/default/props.conf. The Exchange app explicitly sets SHOULD_LINEMERGE to false in the fwd_* apps. Currently, the deployment doc for the Exchange app states that fwd_* components should be pushed out to the Universal Forwarders, Indexers and Search Heads. It does not mention Heavy Forwarders that will receive and send Exchange data.

To correct the problem, fwd_* components should also be pushed to the Heavy Forwarders. This can be accomplished with your deployment server or manually. The props.conf files in the fwd_* components will set SHOULD_LINEMERGE to false for all of the Exchange sourcetypes. Once this goes into effect on the Heavy Forwarders, the User counts will be correct again.