Getting Data In

Why is some of the data that is coming in to Splunk via UDP getting dropped?

caili
Path Finder

Every UDP packet is like this below:

<headinfo product="wf" hash="D95F-7C1A-0F4D-A311" msgtype="3840" sip="0"/>
<wsec dev_ip="59.57.13.68"  protocol_type="HTTP"  src_port="57681"  protect_id="2395196206"  wa_host="www.shis.gov.cn"  url="L3NzemZ3bXMvd3d3cm9vdC9pbmeC5qc3A="  count_num="1"  agent="aHR0cCUyZGZveGJv"  src_ip="59.6.74.114"  http_protocol="HTTP/1.1"  dst_port="80"  dst_ip="59.60.3.146"  stat_time="1463656155"  wa_referer="Tm9uZQ=="  method="2"  alertlevel="3"  event_type="28"  />

There are two lines, but I want to combine to one event

Splunk indexes the data like this:

<wsec dev_ip="59.57.13.68"  protocol_type="HTTP"  src_port="57681"  protect_id="2395196206"  wa_host="www.shis.gov.cn"  url="L3NzemZ3bXMvd3d3cm9vdC9pbmeC5qc3A="  count_num="1"  agent="aHR0cCUyZGZveGJv"  src_ip="59.6.74.114"  http_protocol="HTTP/1.1"  dst_port="80"  dst_ip="59.60.3.146"  stat_time="1463656155"  wa_referer="Tm9uZQ=="  method="2"  alertlevel="3"  event_type="28"  />

the first line is dropped.

What can I do for this?
Thanks in advance.

Tags (2)
0 Karma

jkat54
SplunkTrust
SplunkTrust

If the first line is indexed but not as the same event, then try this in your props to combine the events:

[sourcetypeName]
...
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = <headinfo

If the first event is being dropped altogether, then this may be due to UDP's connection-less design. It could also be due to other options set in your props & transforms.

0 Karma

caili
Path Finder

The first line is missing, not be indexed

if I add BREAK_ONLY_BEFORE = <headinfo in props.conf,then all the data(less than 256) will be combined as one envent

the configuration of props.conf is :

[a_log]
category = Custom
description = receiving UDP port data 
pulldown_type = 1
#SHOULD_LINEMERGE = true
#BREAK_ONLY_BEFORE = <headinfo

There is no transforms.conf in local directory

0 Karma

jkat54
SplunkTrust
SplunkTrust

Did you try it and all data less than 256 characters was combined into one event, or are you saying thats what will happen if you do try it?

should <headinfo... be one event and <wsec... be another event or do you want all of these to be one event?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...