python 3.5 - asyncio - running a concurrent pool of connections

Question

I'm mainly a PHP / Java programmer and I'm fairly new to Pythons concurrency, so I'm having this problem:

I have a websocket server accepting connections on asyncio's main event loop and from what I can tell the event loop is single-threaded, so I'm trying to make a ThreadPoolExecutor with a thread for each connected client that has an endless await-ed loop listening for communication from each connected client

This is my code:

import asyncio
from concurrent.futures import ThreadPoolExecutor

loop = asyncio.get_event_loop()
connected = set()

executor = ThreadPoolExecutor(max_workers=500)

def main():
    import websockets
    from includes.Client import Client

    async def handler(websocket, path):
        global connected
        client = Client(websocket)
        connected.add(client)
        executor.submit(await client.receive_loop())
        print('< client connected')

    server = websockets.serve(handler, 'localhost', 40004)
    loop.run_until_complete(server)
    loop.run_forever()

if __name__ == '__main__' : main()

this is the "client.receive_loop()"

async def receive_loop(self):

    try:
        while True:
            message = await self.socket.recv()
            self.process_message(message)
    except websockets.exceptions.ConnectionClosed:
        print('> client disconnected')
        self.socket.close()

my problem is that instead of running the "receive_loop" on the thread pool it is await-ing for the client to disconnect before the print "client connected" is run. Although, oddly, it accepts multiple connections...

I've tried looking through Python's, Asyncio's and websocket lib's documentation but I haven't yet understood exactly how to do this, and I find python's documentation to be very confusing (atleast compared to PHP's).

Any help?

Answer 1

In your example receive_loop should not be a coroutine, but a simple function instead. But in general what you are trying to do doesn't makes much sense. Adding a thread for every client will not make it any better. Asyncio can handle all the clients in one thread. On the other hand you can do it only with threads without using asyncio and coroutines at all. So it's more like you need to decide - are you going to use threads or asyncio here. And I would suggest asyncio, because in python threads doesn't add much value because of GIL. Also I would suggest to check aiohttp library for asyncio, it has examples for websockets.