Why is our third party logstash only receiving half of logs forwarded from Splunk?

[thezero]

Hi Team,

We are currently forwarding Windows logs to third party siem and logstash but there is problem. Looks like third party receiving receiving only 50% of logs although we are forwarding all logs. Firewall rules are in place to forward and receive logs.

Data flow is as below:

Splunk Universal forwarder --->Splunk HWF ---->Third party using UDP via syslog.

We are using below config:

outputs.conf

[tcpout:syslog]
server = destination host:port

props.conf

[windows]
TRANSFORMS-forward = windows

transforms.conf

[windows]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=syslog

Am I missing something?

What difference will it make if i add below config?

sendCookedData=false.

Are there any limitations on how much data we forward via UDP? We are trying to send almost few MBlogs per second.

There are no errors in splunkd logs or metrics.log

Please advise.

[koshyk]

in your props.conf, [windows]
I hope that's something you defined? can u ensure that [windows] contain all the eventogs? [WinEventLog:Security], [WinEventLog:Application], [WinEventLog:System] etc..

[thezero]

HI Koshyk,

Thanks for the suggestion.I have added [WinEventLog:Application], [WinEventLog:System] to config but their overall count as comapare dto security logs is 1% only..so issue stil continues i.e. stilwe are receiving 505 logs at third party 😞

[dwaddle]

Are you sure this is UDP syslog? It looks more like TCP. I would not be surprised if the HWF's Output Queues are becoming full, blocking, and dropping some data.

[thezero]

Hi Dwaddle,

Yes. I am sure it's UDP syslog.At the destination we are listening for UDp traffic only.We are receiving the UDP traffic at destination.The problem is we receiving approximately 50% of logs .I also searched for any logs with keyword blocked=true (metric.log in SPlunk),but no results.

Regards,
Thezero