How to process data stream from multiple client sockets without closing any of those connections?

Question

I have a server which accepts a stream of JSON data from a client socket. The server needs to keep all the connections open forever and process the data that comes from them, but the current version of my code can handle a single connection only because the second while loop never ends once the first connection has been established.

import socket

IP = '127.0.0.1'
PORT = 1235
BUFFER = 10
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind((IP, PORT))
server_socket.listen(10)
print(f"Listening for incoming connections @ {IP}:{PORT}")


while True:
    client_socket, client_address = server_socket.accept()
    print(f"Established connection with {client_address}")
    while True:
        received_message = client_socket.recv(BUFFER).decode('utf-8')
        while '}' not in received_message:
            received_message += client_socket.recv(BUFFER).decode('utf-8')
        print(client_address, received_message)
        client_socket.send("Message received".encode('utf-8'))

Answer 1

Was able to solve the problem by implementing threading. Here's the working code if anyone wants to use it.

import socket
from _thread import *

IP = '127.0.0.1'
PORT = 1235
BUFFER = 100
thread_count = 0
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind((IP, PORT))
server_socket.listen(10)


def threaded_client(client_socket, address):
    while True:
        received_message = client_socket.recv(BUFFER).decode('utf-8')
        while '}' not in received_message:
            received_message += client_socket.recv(BUFFER).decode('utf-8')
        print(client_address, received_message)
        client_socket.send("Message received.".encode('utf-8'))


while True:
    client_socket, client_address = server_socket.accept()
    print(f"Established connection with {client_address}")
    start_new_thread(threaded_client, (client_socket, client_address))
    thread_count += 1
    print(thread_count)

Answer 2

The primary problem you're facing is that almost all of python's IO is blocking . That means that the execution of the program stops when you want to read or write to a file, or to a socket in this case. The standard solution is to use threads through the threading -library, but there are many other options.

I'd recommend watching David Beazley's video on concurrency , which covers this topic quite well, It can get a little bit complicated, but that's because it's covering a lot of ground fairly quickly!