Getting Data In

Why is some of the data that is coming in to Splunk via UDP getting dropped?

caili
Path Finder

Every UDP packet is like this below:

<headinfo product="wf" hash="D95F-7C1A-0F4D-A311" msgtype="3840" sip="0"/>
<wsec dev_ip="59.57.13.68"  protocol_type="HTTP"  src_port="57681"  protect_id="2395196206"  wa_host="www.shis.gov.cn"  url="L3NzemZ3bXMvd3d3cm9vdC9pbmeC5qc3A="  count_num="1"  agent="aHR0cCUyZGZveGJv"  src_ip="59.6.74.114"  http_protocol="HTTP/1.1"  dst_port="80"  dst_ip="59.60.3.146"  stat_time="1463656155"  wa_referer="Tm9uZQ=="  method="2"  alertlevel="3"  event_type="28"  />

There are two lines, but I want to combine to one event

Splunk indexes the data like this:

<wsec dev_ip="59.57.13.68"  protocol_type="HTTP"  src_port="57681"  protect_id="2395196206"  wa_host="www.shis.gov.cn"  url="L3NzemZ3bXMvd3d3cm9vdC9pbmeC5qc3A="  count_num="1"  agent="aHR0cCUyZGZveGJv"  src_ip="59.6.74.114"  http_protocol="HTTP/1.1"  dst_port="80"  dst_ip="59.60.3.146"  stat_time="1463656155"  wa_referer="Tm9uZQ=="  method="2"  alertlevel="3"  event_type="28"  />

the first line is dropped.

What can I do for this?
Thanks in advance.

Tags (2)
0 Karma

jkat54
SplunkTrust
SplunkTrust

If the first line is indexed but not as the same event, then try this in your props to combine the events:

[sourcetypeName]
...
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = <headinfo

If the first event is being dropped altogether, then this may be due to UDP's connection-less design. It could also be due to other options set in your props & transforms.

0 Karma

caili
Path Finder

The first line is missing, not be indexed

if I add BREAK_ONLY_BEFORE = <headinfo in props.conf,then all the data(less than 256) will be combined as one envent

the configuration of props.conf is :

[a_log]
category = Custom
description = receiving UDP port data 
pulldown_type = 1
#SHOULD_LINEMERGE = true
#BREAK_ONLY_BEFORE = <headinfo

There is no transforms.conf in local directory

0 Karma

jkat54
SplunkTrust
SplunkTrust

Did you try it and all data less than 256 characters was combined into one event, or are you saying thats what will happen if you do try it?

should <headinfo... be one event and <wsec... be another event or do you want all of these to be one event?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...