Non-blocking sockets multi-threaded receive model

Question

I'm working on an multi-threaded server application for learning puposes. My problem right now is receiving the data.The first time I wrote the application I used WSAAsyncSelect, but I didn't like how it was working ( the window dependency was stupid in my opinion, even if you hid the window.). So I re-wrote it and now I have a thread that goes through the connected clients and checks if there is any data to be be received and pass it to the worker threads. This works for a small ammount of clients, but I think that for a larger amount it might delay other clients too much. A solution I've read about is having a thread for each client, but there are thread limitations. Another solution would be IOCP ( Windows ), but I need to find a good documentation, since the examples I found were a bit too ambigous ( I might be the problem here )

The language I'm using C/C++ under Microsoft Visual Studio 2013 ( WinSock, but I would like to write it in a multiplatform way )

Answer 1

If one thread is enough to handle all your clients, consider having N threads and distribute the clients (eg through hashing or in order of reception) among the available threads.

The key is that the overall number of threads (first in your process: polling+worker... but also in the whole system) must remain constant and not exceeding the number of processing resources (CPUs/cores)

This distinction between polling thread(s) and worker threads is the right way to go. Decoupling using queue structures, freeing the polling threads to do their job (ie polling) unhindered from the (arbitrary) logic running on the worker threads.

Answer 2

On Windows, IOCP is the standard model for scalable async IO. It solves all the problems you mention. It has a sane programming model (with a few API design errors). I believe there are self-contained samples. To learn IOCP I'd create a very simple chat server using this technology and try to get it absolutely right.

The select has the disadvantage that it does not scale (as you said).

If you want portability, look into async IO libraries like Boost asio. They use an IOCP-like model on all modern platforms. They are callback-driven.