Handling multiple requests on a TCP/IP Socket

Question

Is it quite easy to handle multiple requests on a Java TCP / IP socket. Simply accept a message and spawn a thread. Accept another message and spawn another thread. The thing is once you start spawning threads things get more non deterministic. Say you have 10 clients and one client keeps firing requests and the other 9 nine who send requests at 10% percent of the hyperactive client, find it harder to get a look in.

One way you could handle this is have a hashmap of semaphores in your server where every client has a corresponding semaphore. Before you handle a request for any client, you could make it go thru its semaphore and configure the semaphores so that each client could only have a certain number of requests at any one time.

At this stage,I'm thinking yeah that works but is there a better way or a library that does this?

Answer 1

... but is there a better way ...?

I use one accepting-thread per serversocket and a pool of pre-spawned threads to handle the workload. The accepting-thread only accepts connections (does nothing else) and gives the handler-socket to one of the threads in the pool. That thread then works with the handler-socket until the client is done, then closes the handler-socket.

You can scale-out this setup as far as you like: If you notice that the accepting-thread is waiting for pool-threads most of the time then you need to x2 your number of pool-threads, if you notice that the accepting-thread is the bottle-neck you create both (A) another accepting-thread and (B) another socket from which it accepts connections and optionally (C) put these on another machine.

The specific problem you are describing with the one hyper-active client can be intended/desired if the client is more important than others: in which case you have to do nothing. Or it can be considered a denial-of-service attack, in which case you should have a heuristic that just disconnects the client and temporarily bans its ip-address.