Reliable UDP implementation design issue

Question

I have been working on a customization around UDP to make it reliable. I have this design problem which I realized only after my entire program was ready and I started sending packets from source to sink.

Scenario: I created a single thread for reception of packets. The parent does packet sending job. Since this is just a POC, I have kept the buffer and common data structures as global pointer for whom memory is allocated on heap by the parent. I am taking care of critical memory sections using mutex.

As part of reliability I send across some control packets apart from data packets. At anytime, client will send data packets and receive control packets from server whereas server will receive data packets and send out control packets. I have used single socket, as my understanding is send & recv works simultaneously on single socket and default blocking.

Problem: For test purpose, I send 100 packets from source to sink. Unfortunately, the thread on the server side busy keeps receiving packets and stores it in buffer. Server code isn't delivering packets to application until the parent thread gets the context switch. This add to unacceptable delay in overall communication.

Please help me understand, what is the issue; what can be changed to improve the performance?

Thanks in advance, Kedar

Answer 1

Since you're using a mutex, when the mutex is released on one thread after the packets are sent, then the other thread should consume the packet. Perhaps you are not releasing the mutex soon enough.

Alternately, let the socket's select() method handle the unblock-on-receive for you.