My NetFlow generator captures both ends of an HTTP session and provides the metadata to Splunk via the Splunk Add-on for IPFIX. (Splunk_TA_ipfix). The issue I'm experiencing is that each side of the connection appears as a discrete event. For example (sanitized data ahead):
4/28/17
10:48:29.000 PM
Sequence="408451"; Template="568"; destinationIPv4Address="10.20.30.40"; protocolIdentifier="6"; sourceTransportPort="80"; destinationTransportPort="58820"; tcpDestinationPort="58820"; tcpSourcePort="80"; udpDestinationPort="0"; udpSourcePort="0"; sourceIPv4Address="5.4.3.2"; HttpRspStatus="200"; HttpReqUrl="";
destinationIPv4Address = 10.20.30.40 eventtype = netflow sourceIPv4Address = 5.4.3.2 sourceTransportPort = 80 tcpDestinationPort = 58820 tcpSourcePort = 80
4/28/17
10:48:29.000 PM
Sequence="408450"; Template="568"; destinationIPv4Address="5.4.3.2"; protocolIdentifier="6"; sourceTransportPort="58286"; destinationTransportPort="80"; tcpDestinationPort="80"; tcpSourcePort="58286"; udpDestinationPort="0"; udpSourcePort="0"; sourceIPv4Address="10.20.30.40"; HttpRspStatus="0"; HttpReqUrl="www.example.com";
My goal is to stitch together the full event such that I'm able to see both the HttpReqUrl and the HttpRspStatus as a single entry e.g. HttpReqUrl=www.example.com HttpRespStatus=200. Note that for HTTP request, the response is 0 and for the response, the request is null.
I've tried various evals and transactions, but I've gotten nowhere. Either I end up with too greedy of situation or the events remain apart. I've checked and, unfortunately, the sequence number is not sufficient for stitching (that's not a TCP sequence either, but rather a Netflow sequence - either way, didn't help).
We are a Splunk partner and we provide this functionality in one of our NetFlow Optimizer Logic Modules reporting Host Pairs network conversations. This Module stitches client-server request-response flows, reporting bytes and packets server-to-client and client-to-server in separate fields in one syslog message.
Try NetFlow Optimizer for free by visiting https://www.netflowlogic.com/download/