How to stop a thread in a web server

Question

I have a Java socket.io server, based on netty implementation (of com.corundumstudio.socketio).

The server handles requests using some closed-source code, and sometimes, a request may get stuck indefinitely.

The big problem starts when the requests get stuck inside a synchronized{} block, so that they keep holding a resource, and no other request can be fulfilled.

I would like to add an option to stop/kill a specific thread, so that it doesn't halt the entire application.

Thread.stop() is deprecated, and also apparently doesn't work with netty, as netty catches the ThreadDeath exception and converts it to a harmless warning message...

adding an interrupt flag is impossible because the difficult code is closed.

What is a good way to implement the server such that it is robust on this regard?

Answer 1

I am sure it goes without saying that what you really need to do is figure out and fix what-ever the problem is with the synchronized resource, because killing off threads will not, in the end, actually get the job done. Without knowing anything about the resource, it's impossible to advise you on that specific point.

In the interim, I would suggest you wrap the problematic code segment with a ReentrantLock which each thread must acquire in order to reach the synchronized block. Effectively, you are replacing the synchronized behaviour (from which a thread cannot be pulled out of) with a lock which has the same guarding properties, but for which you can timeout the acquiring threads (so the threads don't pile-up and stop your system) by having them acquire the lock with a tryLock(long timeout, TimeUnit unit) with an appropriate timeout. The ReentrantLock also has a layer of "instrumentation: allowing you to determine how many threads are waiting on the lock, and acquire a direct reference to the threads themselves. That last point will be useful to you as it allows you to interrupt the waiting threads as an alternative to * stop *ping them. Once a waiting thread is interrupted, it will throw an interrupted exception and then continue on its way in a perfectly legal and non-deprecated way.

At the risk of raising the ire of some purists, and with some caution advised, you could have other threads avoid entering the synchronized block by using sun.misc.Unsafe.tryMonitorEnter which will return false if the synchronization point is already blocked.