Why Java DatagramSocket doesn't receive all udp packet that clients have sent?

Question

my clients send udp packets with high rate. i'm sure that my java app layer doesn't receive all udp packets that clients sent becuase the number of recieved packets in wireshark and my java app doesn't match. because wireshark receive more udp packets so i'm sure udp packets didn't lost in network.

the code is here:

receive packets in a thread and offer to a LinkedBlockingQueue and on another thread consume take packets from LinkedBlockingQueue and then call onNext on a rx-java subject.

socket = new DatagramSocket(this.port);
socket.setBroadcast(true);
socket.setReceiveBufferSize(2 * 1024 * 1024);

// thread-1
while (true) {
  byte[] bytes = new byte[532];
  DatagramPacket packet = new DatagramPacket(bytes, bytes.length);
  try {
    this.socket.receive(packet);
    queue.offer(
        new UdpPacket(
            packet.getPort(), packet.getAddress().getHostAddress(), packet.getData()));
  } catch (IOException e) {
    e.printStackTrace();
  }
}


// thread-2
UdpPacket packet;
while ((packet = queue.take()) != null) {
     this.receiveMessageSubject.onNext(packet);
}

Host OS: Ubutnu 18.04

Answer 1

Very difficult to give a straight answer but from my experience with UDP message processing in Java it really matters to improve performance of processing the messages , especially with large volumes of data.

So here are some things that I would consider:

1) You are correct to process UDP messages on a different queue. But, the queue has a limited size. Do you manage to process messages fast? Otherwise, the queue fills up and you are blocking the while loop. Some simple logging there could let you know if this is the case. Putting them on a queue where they can be pooped out on a different step is awesome but you also need to make sure that the processing is fast as way and that the queue does not fill up.

2) Are all your data-grams less than 532 bytes? Maybe some loss occurs due to larger messages that don't fill the buffer.

Hope this helps,

Answer 2

I had a similar issue to this recently in a different language. I'm unsure if it works the same in Java, but this may be helpful to you.

So as data packets come into the socket, they are buffered and you have set your buffer size, but you are still only reading a single data packet, even though the buffer could be holding more. As you're processing one datagram at a time, your buffer is filling up even more and eventually when its full, data could be lost as it can't store any more datagrams.

I checked the documentation for DatagramSocket

Receives a datagram packet from this socket

I'm unsure on the functions you would need to call in Java, but here's a little snippet that I am using.

while (!m_server->BufferEmpty()) {
        std::shared_ptr<Stream> inStream = std::make_shared<Stream>();
        std::vector<unsigned char>& buffer = inStream->GetBuffer();

        boost::asio::ip::udp::endpoint senderEndpoint = m_server->receive(boost::asio::buffer(buffer),
            boost::posix_time::milliseconds(-1), ec);

        if (ec)
        {
            std::cout << "Receive error: " << ec.message() << "\n";
        }
        else
        {
            std::unique_ptr<IPacketIn> incomingPacket = std::make_unique<IPacketIn>();
            incomingPacket->ReadHeader(inStream);

            m_packetProcessor->ProcessPacket(incomingPacket, senderEndpoint);

            incomingPacket.reset();
        }

        ++packetsRead;

        inStream.reset();

}

This basically says that if the socket has any data for the current frame in its buffer, keep reading datagrams until the buffer is empty.

Unsure on how the LinkedBlockingQueue works, but this could also be causing a bit of a problem if both threads are trying to access it at the same time. In your UDP reading thread you could be blocked for some time, and then packets could be received during this time.