Python TCP server accepts new socket, but never receives data

Question

I'm at the end of my rope - I can't figure this one out. I'm pretty new at Python and I'm writing up a TCP Server / Client application.

The server listens for connections and spawns off a thread to handle the connection. The communication is very simple - Client makes connection, sends message, server responds and then closes the connection (ala HTTP)

The problems is that my server code below does not consistently receive any data at all from the client. The server accepts the new connection, spins up the new thread, and then blocks on the .recv method.

Server Listener

class TCPListener(threading.Thread):

    def __init__(self, ip = settings.BIND_IP, port = settings.BIND_PORT):
        super(TCPListener, self).__init__()
        self.daemon = True
        self._port = port
        self._ip = ip
        self.stop = threading.Event()
        self.stop.clear()

        self.tcp_server_socket = socket(AF_INET, SOCK_STREAM)
        self.tcp_server_socket.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
        self.tcp_server_socket.setblocking(False)
        self.tcp_server_socket.bind((self._ip, self._port))

    def run(self):
        # Listen for client connection requests
        with lock:
            utils.log_message("Listening for ledger messages on port {0}".format(self._port))

        try:
            self.tcp_server_socket.listen(5)

            # List for managing spawned threads
            socket_threads = []

            # Non-blocking socket loop that can be interrupted with a signal/event
            while True and not self.stop.is_set():
                try:
                    client_socket, address = self.tcp_server_socket.accept()

                    # Spawn thread
                    client_thread = TCPConnectionThread(client_socket)
                    client_thread.start()
                    socket_threads.append(client_thread)

                except Exception as e:
                    continue

            # Clean up all the threads
            for thread in socket_threads:
                thread.join()

        except Exception as e:
            print("Could not bind to port: {0}".format(e))
        finally:
            self.tcp_server_socket.close()

Server Connection Handler

class TCPConnectionThread(threading.Thread):

    def __init__(self, socket):
        super(TCPConnectionThread, self).__init__()
        with lock:
            utils.log_message("Spawning TCP Connection Thread from {0}".format(socket.getsockname()))
        self._socket = socket


    def run(self): 
        # Get message
        message = ''
        data = True
        while data:
            data = self._socket.recv(4096)  #BLOCKS HERE 99% OF THE TIME
            message+=data.decode()

        with lock:
            utils.log_message("Received message from {0}:\n{1}".format(self._socket.getsockname(), message))

        self._socket.sendall(response)
        self._socket.close()

The thread that handles the connection from my client just blocks. If I put a breakpoint on that point (socket.recv), it will probably receive the data, but if I run it normally it blocks indefinitely. I'm not sure why debugging it would affect it, perhaps introducing a delay? I can see in wireshark that the client is sending the message and the server ACKs it, but it is never returned from recv().

Any help would be greatly appreciated. I have wasted far too much time on what is most likely a simple bug.

Answer 1

self._socket.recv is a blocking call. It will block until it receives data on the socket. I think the issue is :

data = True
while data:
            data = self._socket.recv(4096)  #BLOCKS HERE 99% OF THE TIME
            message+=data.decode()

Server receives data but again gets into the loop and waits for further data. What is the criteria for exiting the loop? If the client sends data but does not disconnect, the loop will always be active. Only if the client disconnects will the loop break.

Answer 2

I believe you're making things overly complicated for yourself. See the following example from the Python Cookbook :

from socket import AF_INET, SOCK_STREAM, socket
from concurrent.futures import ThreadPoolExecutor

def echo_client(sock, client_addr):
    '''
    Handle a client connection
    '''
    print('Got connection from', client_addr)
    while True:
        msg = sock.recv(65536)
        if not msg:
            break
        sock.sendall(msg)
    print('Client closed connection')
    sock.close()

def echo_server(addr):
    print('Echo server running at', addr)
    pool = ThreadPoolExecutor(128)
    sock = socket(AF_INET, SOCK_STREAM)
    sock.bind(addr)
    sock.listen(5)
    while True:
        client_sock, client_addr = sock.accept()
        pool.submit(echo_client, client_sock, client_addr)

echo_server(('',15000))

In the very least this :

    while True:
        msg = sock.recv(65536)
        if not msg:
            break

should take care of your blocking issue.