Is there any way we can discard unwanted log events at universal forwarder level

[usha_nittala]

Hi All,

We can do selective forwarding at heavy forwarder or indexer level but i want to know with latest version , is there any possibility of discarding unwanted events in logs ( based on regex pattern) at universal forwarder level. We dont have heavy forwarders so putting lot of conditions in transforms will increase the parsing load on indexer. So want to know if we can configure anything on universal forwarder itself.

Thanks in advance.

[FrankVl · Accepted solution]

The basic answer is no, this is specifically one of those use cases you would use Heavy Forwarders for.

Some input methods (e.g. winevtlog) do offer whitelist/blacklist settings though, so you might want to take a look at what type of inputs you have and then check the inputs.conf spec to see if that offers any options.

Also, if you are receiving syslog data, you could use a syslog daemon to do the pre-processing and drop some of the data.

[masonmorales]

Upvoted, as this is the correct answer. A universal forward is not capable of performing event-level parsing/filtering operations (aside from Windows Event ID). For a good write-up on UF vs HF, check out: https://www.splunk.com/blog/2016/12/12/universal-or-heavy-that-is-the-question.html

[sloshburch]

Also keep in mind that if you end up using a HW you will be causing the data to get more condensed than if it were done at the indexer layer.

Are you actually seeing load issues on the indexers? Before creating more complexity in the environment, we should make sure this is the right feature to play with.

[usha_nittala]

Thanks @FrankVl , I will check whitelist/blacklist settings in inputs.conf spec.

[deepashri_123]

Hey@usha_nittala,

Yes you can filter events before indexing. Refer the link below:
http://docs.splunk.com/Documentation/Splunk/7.0.3/Forwarding/Routeandfilterdatad

Let me know if this helps!!

[FrankVl]

@usha_nittala is looking for a solution on UF, that document is mostly about things you can do on HF.

[usha_nittala]

Thanks @deepashri_123 and @FrankVl
@FrankVl is right. We don't have enough capacity to add heavy forwarders and we have limited number of indexer servers.We did not want to increase load on indexers and hence wanted to find out if any kind of parsing is done at Universal/light forwarder level.