All Apps and Add-ons

Splunk App for Stream: event breaking of UDP packets from same IP

kwchang_splunk
Splunk Employee
Splunk Employee

Hello,

I'm testing Splunk App for Stream with UDP packet generator.
When I used same IP and Mac address, all UDP packets were merged and became a single event in Splunk indexer.
How can I change this behaviour? I'd like to have separated events even when the source ip and mac address values are same.

Thank you in advance.

Tags (1)
0 Karma

amielke
Communicator

My Problem seems to be similar. My customer send every 100ms a new stream off UDP packets. Splunk Stream App is unable to break the events after this time.
I wrote a test client with a thread sleep 100 ms.

I try in the config file streamfwd.xml the configuration: 1
The unit is seconds, I need milliseconds.

Somebody have a after solution or idea for this problem?

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.text.SimpleDateFormat;
import java.util.Date;

class UDPClient {
    public static void main(String args[]) throws Exception {
        System.out.println("Start UDPCLient");
        BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
        BufferedReader durationFromUser = new BufferedReader(new InputStreamReader(System.in));
        InetAddress IPAddress = InetAddress.getByName("192.168.1.10");
        byte[] sendData = new byte[1024];
        byte[] receiveData = new byte[1024];
        String sentence = null;
        System.out.println("Test time in Secondes?");
        String timeStr = durationFromUser.readLine();
        long durationInMs = 120000L;
        if (timeStr != null && !timeStr.isEmpty())
            durationInMs = 1000L * Long.parseLong(timeStr);
        long starttime = System.currentTimeMillis();
        long duration = starttime + durationInMs;
        SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.S");
        String startTimeStr = df.format(new Date(starttime));

        System.out.println("Starttime: " + startTimeStr + " Duration: " + durationInMs);
        DatagramSocket clientSocket = new DatagramSocket();
        int i = 0;
        boolean stoptimeOK = false;
        long controltime = 0L;
        int byteSize = 0;
        while (!stoptimeOK) {
            sentence = "UDP_Msg_";
            sentence = sentence.concat("" + df.format(new Date(System.currentTimeMillis())) + " ");
            sendData = sentence.getBytes();
            byteSize = byteSize + sendData.length;
            DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, 9875);
            clientSocket.send(sendPacket);
            Thread.sleep(100);
            controltime = System.currentTimeMillis();
            if (controltime >= duration) {
                stoptimeOK = true;
            }
        }
        clientSocket.close();
        String finishTime = df.format(new Date(controltime));
        System.out.println("EndeTime:  " + finishTime + " ByteSize " + byteSize);
        System.out.println("Finish");
    }
}
0 Karma

amielke
Communicator

If I user the UDP input the packets are correct readed, each 100 ms. But only if I send direct to Splunk. If I transmission between source and sink and I hear on the channel, the UDP packet is not recognized correctly.

0 Karma

vshcherbakov_sp
Splunk Employee
Splunk Employee

Hello kwchang,

Stream generates UDP "flow" events, not individual UDP packet events. UDP flow is defined as a group of packets flowing between the same endpoint until it times out. You can control the timeout by setting TcpConnectionTimeout parameter in streamfwd.xml file (never mind the "TCP" part in the name, it should really be renamed to "FlowTimeout" as it work for both TCP and UDP flows) to a lower value, but it still won't give you an event per single UDP packet.

A couple of questions: what do you expect to see in a single UDP packet event - bytes_in/out, etc. or something more specific? What is the application layer protocol transported via UDP?

0 Karma

kwchang_splunk
Splunk Employee
Splunk Employee

Thank you for your answer.

My customer is doing a performance test. About 1.8Gbps UDP traffic with very small(128~1024bytes) and meaningless payload is being generated. So.. if each of packet has same source ip and mac address, according to your advice, I will have a single 240MB sized event in every seconds with "TcpConnectionTimeout = 1" setting.

What is the minimum configurable value for TcpConnectionTimeout? When I tested with 0 or 0.1, it didn't seem to work.

And would you please give some advice on my previous email to you? Sent to your corporate email.
Thank you in advance.

0 Karma

kwchang_splunk
Splunk Employee
Splunk Employee

I changed the test scenario to use random src ip address. And it works. Thank you.
And please find my previous email to your corp email account. Please also give your advice on that. Thank you so much.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...