Before the question, a bit of background.
I have a setup in which I have two machines.
The first collects data from various devices and sends it directly to the second over UDP, targeting port 5005.
The second is running Splunk and has been configured to listen on port 5005 for UDP messages, and record them on a "sandbox" index.
These machines are isolated from the internet and are connected by Ethernet cables to a hub switch right next to them.
For the sake of brevity, let's call them M1 and S, short for Machine 1 and Splunk Machine.
S has been assigned IP 192.168.0.5 while M1 has 192.168.0.6.
There are three ways I can transmit packets from M1 to S.
The first is by running from the terminal on S1:
echo -n "{'Message':'hello'}" > /dev/udp/192.168.0.5/5005
This message is successfully sent from M1 to S and shows up in the sandbox index.
The second is by running some scripts that emulate our desired behavior and form packets using Scapy, again on M1 targeting S.
This is Scapy, not Scipy.
This process also successfully completes the full loop and shows up in the sandbox index.
The final method, and the one this question centers on, is to open scapy on M1 and generate packets and send them that way. Assuming we want to emulate sending packets from a docker container on M1 with an an IP of 10.10.12.9, the command used to generate these packets is as follows, with some slight editing, namely substituting text for the actual mac addresses and placing each field on its own line:
sendp(Ether(dst="<mac address of container here>",src="<mac address of S here>")
/IP(src="10.10.12.9",dst=192.168.0.5")
/UDP(dport=5005,sport=33017)
/Raw(load="{'Message':'Hello.'}"), iface="veth201")
If I execute this command in Scapy, I'm told that it sends the packet.
If I run a tcpdump on S, I can see that the packet generated by the command does in fact travel from M1 to S and is received.
However, this packet is entirely ignored by Splunk.
When I say "entirely ignored" I mean that, if I open the Splunk Web Search in a browser on S, and start a real time search with a 5 minute window for all events on the "sandbox" or "main" indexes, sending the scapy packet as described above does not cause an event, while the other two methods trigger events on the "sandbox" index as expected.
How do I make Splunk recognize this packet?
EDIT 1: I've found that if I run
send(IP(src="192.168.0.6",dst=192.168.0.5")
/UDP(dport=5005,sport=33017)
/Raw(load="{'Message':'Hello.'}"), iface="veth201")
Then the packet will be received successfully in TCP dump and is recognized by Splunk.
If I then change the command to
send(IP(src="10.10.12.9",dst=192.168.0.5")
/UDP(dport=5005,sport=33017)
/Raw(load="{'Message':'Hello.'}"), iface="veth201")
Then it will fail once again to be recognized by Splunk, but the packet is successfully delivered according to tcpdump.
... View more