Splunk Search

Splunk Stream split separate events for persistent tcp stream or Disable TCPREASSEMBLY or some other method

pinakicybermak
New Member

Hi Everyone,

I am using splunk stream. Packet stream to capture data from source and destination content fields.
For a persistent TCP connection i just cannot seem to break/split in separate events or lines.
Is there no way to so this?
Other advice appreciated.
I am willing to check other alternatives, willing to truncate the data, etc etc etc

Sample event Single TCP Conn Open/Close
{"endtime":"2017-09-17T15:30:47.271015Z","timestamp":"2017-09-17T15:30:36.440073Z","ack_packets_in":4,"ack_packets_out":5,"app":"tcp","bytes":645,"bytes_in":353,"bytes_out":292,"client_rtt":16,"client_rtt_packets":1,"client_rtt_sum":16,"connection":"192.168.100.3:65534","data_packets_in":1,"data_packets_out":0,"dest_ip":"192.168.100.3","dest_port":65534,"duplicate_packets_in":0,"duplicate_packets_out":0,"missing_packets_in":0,"missing_packets_out":0,"network_interface":"lo0","packets_in":6,"packets_out":5,"protocol_stack":"ip:tcp:unknown","server_rtt":40,"server_rtt_packets":2,"server_rtt_sum":81,"src_ip":"192.168.100.3","src_port":51448,"tcp_status":0,"time_taken":10830958,"SRCCNT":"68656c6c6f"}

Sample Event TCP persistent Stream
{"endtime":"2017-09-17T15:32:06.278243Z","timestamp":"2017-09-17T15:30:57.342570Z","ack_packets_in":3,"ack_packets_out":158,"app":"tcp","bytes":18484,"bytes_in":9624,"bytes_out":8860,"client_rtt":14,"client_rtt_packets":1,"client_rtt_sum":14,"connection":"192.168.100.3:65534","data_packets_in":153,"data_packets_out":0,"dest_ip":"192.168.100.3","dest_port":65534,"duplicate_packets_in":0,"duplicate_packets_out":0,"missing_packets_in":0,"missing_packets_out":0,"network_interface":"lo0","packets_in":157,"packets_out":158,"protocol_stack":"ip:tcp:unknown","server_rtt":33,"server_rtt_packets":154,"server_rtt_sum":5226,"src_ip":"192.168.100.3","src_port":51475,"tcp_status":0,"time_taken":68935687,"SRCCNT":"68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f"}

I need the above stream to be broken up into separate events. Sorta like a wireshark view

Thank You & appreciate any and all ideas/assistance.
Pinaki

Tags (1)
0 Karma

pinakicybermak
New Member

Hi everybody

After extensive research & self learning about the capabilities & limitations splunk, this is not feasible.
I have opted to use Protocol Data Input app.
Using Module Inputs I have managed to get the results.

Again thank you everyone for your support and answers.

Pinaki

0 Karma

Richfez
SplunkTrust
SplunkTrust

Pinaki,

If I am reading this right, your problem is that Splunk/Stream reassembled the packets in a TCP flow, but you want to see the raw individual packets that made up that flow?

Have you tried faking it out by setting maxTcpReassemblyPacketCount in streamfwd.conf to 1 instead of the default 500000?

I do NOT know if that will work - well, I know it should work but I don't know if it will work usefully. It could very well just make a mess of things. 😞

Give that a try, though, and report back. If that does the trick I would be happy to convert this to an answer so you can accept it. If it does not, well, it may be a useful comment/trick for the next person who stumbles across this question.

Happy Splunking!
-Rich

0 Karma

pinakicybermak
New Member

Dear Rich,

Thank you for your reply. Yes i have already tried that. It does work, but again not a perfect solution. If i could configure that to "ms" instead of "sec" the result would be better.

Thank You
Pinaki Banerjee

0 Karma

Richfez
SplunkTrust
SplunkTrust

When you say if you could configure that to "ms" instead of seconds, are you talking about the timestamp that Splunk is using isn't using the full precision available? That might be fixable.

0 Karma

pinakicybermak
New Member

Dear Rich,

Clarification and apologies. I used the tcpConnectionTimeout (lowest value 1 sec) to almost solve the problem but I faced scenarios where I got multiple transactions with 1 sec.
I also used maxTcpReassemblyPacketCount = 1 but i was getting 2 records always.

If I have understood your comment correctly - "ms", "full precision" - This is an idea i havent tried. If this is doable then maybe it will break with full precision timestamp and i may get all the records including the breakdown handshakes etc. I want to try. Can you give me how to do it?

Thank You
Pinaki

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

@pinakicybermak - are all the SRCCNT values identical in length, or is there another way to distinguish where one ends and the next begins?

0 Karma

pinakicybermak
New Member

Dear Dal,

SRCCNT values - Not identical.
SRCCNT values extracted from either dest_content/src_content in hex
no distinguishing start/end
for example 1 hex full packet maybe
"00ac00ef384eeee000000100ac00ef384eeee0000020000000ac00ef384eeee<128Bytes of data><128bytes>"
2nd packet may be
"00ac00ef2343234000000100ac00ef23432340000020000000ac00ef2343234<128Bytes of data><128bytes>"
3rd packet maybe
"00ab00ef1234000000100ab00ef12340000020000000ab00ef1234<128Bytes of data><128bytes>"
I am actually trying to parse ISO8583 msg which comes in hex. I am successful if the data is coming with 1 tcp open/close. When it comes in stream it looks like
SRCCNT = 00ac00ef384eeee000000100ac00ef384eeee0000020000000ac00ef384eeee<128Bytes of data><128bytes>00ac00ef2343234000000100ac00ef23432340000020000000ac00ef2343234<128Bytes of data><128bytes>00ab00ef1234000000100ab00ef12340000020000000ab00ef1234<128Bytes of data><128bytes>

Thank You
Pinaki

0 Karma

Sukisen1981
Champion

I am sorry , is till can not figure this out. You have 49391 in your sample 'would be' event but I can not see that number anywhere in your raw event, where is this number coming from? Also you have 61 , 56 etc, in your screen shot , I can not see them anywhere in your raw event...

0 Karma

lfedak_splunk
Splunk Employee
Splunk Employee

Hey @sukisen1981, thank you so much for contributing to Splunk Answers! 🙂 Please remember to press "add comment" instead of "add answer" when it's part of discussion. (I just converted this answer post to a comment.)

0 Karma

pinakicybermak
New Member

Hi Suki,

The above are just samples. 49391 is the port. 61/56 are the bytes. This is Wireshark.
This is TCP PERSISTENT Stream. TCP PERSISTENT Stream is a TCP connection is always in connected state till Closure by either client/server.

You can see this value in the pic? - 68656c6c6f. In a single TCP OPEN/Close this comes as 1 event in splunk.

However in a Persistent stream this become "68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f" in splunk. I want the similar line event view in splunk as shown in the wireshark image. This is the value I want in single line for TCP PERSISTENT Stream.

So hence "68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f68656c6c6f" should be broken in separate lines with supporting data
e.g. events
68656c6c6f
68656c6c6f
68656c6c6f
68656c6c6f
68656c6c6f
68656c6c6f
68656c6c6f
68656c6c6f
68656c6c6f
68656c6c6f

hope the above clarifies your question. I just need to know if possible or not. If possible how. Thank you for your help.
Pinaki

0 Karma

pinakicybermak
New Member

So let me clarify my objective - I am concerned only with the data i.e. src & dest content (extracted data field SRCCNT) with couple of other fields like RTT , dest/src ip/port timestamp etc.
In splunk stream - TCP persistent connection all the data values club together and the SRCCNT becomes a huge bulk field (due to TCPreassembly at the splunk) and I cannot work with the data. I ultimately need the data to be shown as below - Each as a Single Event

{"endtime":"2017-09-17T15:30:47.271015Z","timestamp":"2017-09-17T15:30:36.440073Z","ack_packets_in":4,"ack_packets_out":5,"app":"tcp","bytes":645,"bytes_in":353,"bytes_out":292,"client_rtt":16,"client_rtt_packets":1,"client_rtt_sum":16,"connection":"192.168.100.3:65534","data_packets_in":1,"data_packets_out":0,"dest_ip":"192.168.100.3","dest_port":65534,"duplicate_packets_in":0,"duplicate_packets_out":0,"missing_packets_in":0,"missing_packets_out":0,"network_interface":"lo0","packets_in":6,"packets_out":5,"protocol_stack":"ip:tcp:unknown","server_rtt":40,"server_rtt_packets":2,"server_rtt_sum":81,"src_ip":"192.168.100.3","src_port":51448,"tcp_status":0,"time_taken":10830958,"SRCCNT":"68656c6c6f"}
alt text
Thanx
Pinaki

0 Karma

Sukisen1981
Champion

not clear what you want...I need the above stream to be broken up into separate events. Sorta like a wireshark view??
Can you give a sample of how you would like the events to be broken up into?

0 Karma

pinakicybermak
New Member

If the above is not at all possible, kindly let me know. I will choose alternate methods. Thanks

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...