sockets with threadpool server python

Question

I have a simple multithreading server, But it creates a new thread for each socket, I don't want to create a lot of threads. My idea is to receive the messages in other way: when the user send a message, it will add the message to a queue of messages and with a threadpool the server will handle these requests.

The simple multithreaded server:

import socket
import threading

class ThreadedServer(object):
    def __init__(self, host, port):
        self.host = host
        self.port = port
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.sock.bind((self.host, self.port))

    def listen(self):
        self.sock.listen(5)
        while True:
            client, address = self.sock.accept()
            client.settimeout(60)
            threading.Thread(target = self.listenToClient,args = (client,address)).start()

    def listenToClient(self, client, address):
        size = 1024
        while True:
            try:
                data = client.recv(size)
                if data:
                    # Set the response to echo back the recieved data 
                    response = data
                    client.send(response)
                else:
                    raise error('Client disconnected')
            except:
                client.close()
                return False

if __name__ == "__main__":
    port_num = input("Port? ")
    ThreadedServer('',port_num).listen()

How can I implement my idea or is there better way to do it?

Answer 1

The question seems to be pretty old but i also stumble upon the same issue while working on the socket server, so here is the below code which you can use to make threaded socket server which doesnt spawn new threads on arrival.

Just to give gist ThreadingMixIn classes is overided with threaded pool.

class ThreadPoolMixIn(socketserver.ThreadingMixIn):
'''
use a thread pool instead of a new thread on every request
'''
# numThreads = 50    
allow_reuse_address = True  # seems to fix socket.error on server restart
def serve_forever(self):
    '''
    Handle one request at a time until doomsday.
    '''
    print('[X] Server is Running with No of Threads :- {}'.format(self.numThreads))
    # set up the threadpool
    self.requests = Queue(self.numThreads)

    for x in range(self.numThreads):
        t = threading.Thread(target = self.process_request_thread)
        t.setDaemon(1)
        t.start()

    # server main loop
    while True:
        self.handle_request()
    self.server_close()

def process_request_thread(self):
    '''
    obtain request from queue instead of directly from server socket
    '''
    while True:
        socketserver.ThreadingMixIn.process_request_thread(self, *self.requests.get())

def handle_request(self):
    '''
    simply collect requests and put them on the queue for the workers.
    '''
    try:
        request, client_address = self.get_request()
    except socket.error:
        return
    if self.verify_request(request, client_address):
        self.requests.put((request, client_address))

And then it is called in ThreadedTCPRequest Handler and override the numThreads parameter :

class ThreadedTCPServer(ThreadPoolMixIn, socketserver.TCPServer):
#Extend base class and overide the thread paramter to control the number of threads.
def __init__(self, no_of_threads, server_address, ThreadedTCPRequestHandler):
    self.numThreads = no_of_threads
    super().__init__(server_address, ThreadedTCPRequestHandler)

Ultimately creating the server which serves forever :

def create_multi_threaded_socket(CONFIG, HandlerClass = ThreadedTCPRequestHandler,
    ServerClass = ThreadedTCPServer, 
    protocol="HTTP/1.0"):

server_address = ('', CONFIG.port)
HandlerClass.protocol_version = protocol
# httpd = ServerClass(server_address, HandlerClass)
server = ThreadedTCPServer(CONFIG.no_of_threads, server_address, ThreadedTCPRequestHandler)
sa = server.socket.getsockname()
print("Serving HTTP on {} port : {}".format(sa[0], sa[1]))
server.serve_forever()

I got the sample code from : http://code.activestate.com/recipes/574454-thread-pool-mixin-class-for-use-with-socketservert/

Modified bit according to my need. Hope this helps :) .