Getting Data In

Why is some of the data that is coming in to Splunk via UDP getting dropped?

caili
Path Finder

Every UDP packet is like this below:

<headinfo product="wf" hash="D95F-7C1A-0F4D-A311" msgtype="3840" sip="0"/>
<wsec dev_ip="59.57.13.68"  protocol_type="HTTP"  src_port="57681"  protect_id="2395196206"  wa_host="www.shis.gov.cn"  url="L3NzemZ3bXMvd3d3cm9vdC9pbmeC5qc3A="  count_num="1"  agent="aHR0cCUyZGZveGJv"  src_ip="59.6.74.114"  http_protocol="HTTP/1.1"  dst_port="80"  dst_ip="59.60.3.146"  stat_time="1463656155"  wa_referer="Tm9uZQ=="  method="2"  alertlevel="3"  event_type="28"  />

There are two lines, but I want to combine to one event

Splunk indexes the data like this:

<wsec dev_ip="59.57.13.68"  protocol_type="HTTP"  src_port="57681"  protect_id="2395196206"  wa_host="www.shis.gov.cn"  url="L3NzemZ3bXMvd3d3cm9vdC9pbmeC5qc3A="  count_num="1"  agent="aHR0cCUyZGZveGJv"  src_ip="59.6.74.114"  http_protocol="HTTP/1.1"  dst_port="80"  dst_ip="59.60.3.146"  stat_time="1463656155"  wa_referer="Tm9uZQ=="  method="2"  alertlevel="3"  event_type="28"  />

the first line is dropped.

What can I do for this?
Thanks in advance.

Tags (2)
0 Karma

jkat54
SplunkTrust
SplunkTrust

If the first line is indexed but not as the same event, then try this in your props to combine the events:

[sourcetypeName]
...
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = <headinfo

If the first event is being dropped altogether, then this may be due to UDP's connection-less design. It could also be due to other options set in your props & transforms.

0 Karma

caili
Path Finder

The first line is missing, not be indexed

if I add BREAK_ONLY_BEFORE = <headinfo in props.conf,then all the data(less than 256) will be combined as one envent

the configuration of props.conf is :

[a_log]
category = Custom
description = receiving UDP port data 
pulldown_type = 1
#SHOULD_LINEMERGE = true
#BREAK_ONLY_BEFORE = <headinfo

There is no transforms.conf in local directory

0 Karma

jkat54
SplunkTrust
SplunkTrust

Did you try it and all data less than 256 characters was combined into one event, or are you saying thats what will happen if you do try it?

should <headinfo... be one event and <wsec... be another event or do you want all of these to be one event?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...