TCP server hangs, not responding to SYN

Question

I have a strange issue with a TCP server that sometimes hangs. The weird issue is that when it hangs it does not receive any new connection, ie doesn't respond to the initial TCP SYN packet. I was pretty sure that since TCP handshakes are handled by the kernel, even when a program hangs clients should still at the very least receive the initial SYN,ACK. If anyone knows a situation where a program can hang in a way that prevents the OS from even completing the TCP handshake (and without it ever closing the listening socket) please let me know.

PS The program is written in C++ and the OS is Windows Server 2016.

Answer 1

Most likely, the listen queue is full. Not responding to the initial SYN causes the other side to try another SYN a bit later. With luck, the listen queue won't be full at that time. The program is probably not calling accept (or some similar function) often enough.

It's also possible that the program is using the selective accept functionality (see the lpfnCondition parameter to WSASelect ) to choose not to respond to this connection attempt.