How can I effectively test (unit/integration) concurrent code in Java?

Question

I saw this post on SO already but it still begs the question, at least for Java. It seems that this should be a pressing issue for any app (i'm testing a web app for this use case) that's multi-threaded. How have you guys dealt with it in a way that allows your threads to interleave -- inherently meaning testing random thread behavior.

Answer 1

I think it's inherently difficult, since you're looking to prove the absence of something in a (possibly) non-determinant system. Having said that, it's always worth writing repeatable tests to stress your multi-threaded components. If a problem occurs, you may not be able to repeat it, but you can perhaps find the bug by inspection. So make sure your tests log the relevant exceptions in great detail (and possibly input parameters etc.)

It's also worth running a static code analyser. These will pick up (amongst other things) inconsistent synchronisation of variables, and immediately highlight areas for concern. FindBugs is one such tool.

Answer 2

I've typically spawned a huge number (like 100) threads in a unit test and sent them off against it in hopes of hitting a race condition :). Sadly, that's not very definitive. You can improve your odds though by having debug lines in your thread sensitive areas that cause the active thread to sleep for X millis. This strategy allows you to leaving gaping windows where other threads can interleave. These lines would only be active during unit testing (use aspects or hard code lines that only activate when a debug flag is set).

Proving a negative is hard to impossible. You can't really prove you don't have a concurrency issue. All you can do is set up your tests to exercise it as thoroughly as possible.

Answer 3

I suggest you get Java Concurrency in Practice -- not only does it tell you how to avoid race conditions in the first place, there is a section on testing concurrency.

Also there are static analysis tools for concurrency available for Java -- unfortunately while I know they are out there, I have not been in a position to evaluate them.

Answer 4

One good way to test this is to make sure you have access to a multi-cpu machine, then run your test for as long a time/with as many iterations as possible. For example if you have a multithreaded producer consumer algorithm, fill the queue with a good sized list and use 2x as many threads as you might in production. This should at least give you a chance of hitting a race condition or synchronization issue more commonly. Then run this test in a daily build.

Answer 5

My answer to this is to avoid creating situations where you can have race conditions with judicious use of smart design and existing technology. For instance, if your MT app uses self contained work items, you can embed a JMS broker in your application to use as a communication mechanism between threads. Similarly with system resources that might be subject to race conditions, don't make those parts multi-threaded. If you have files you need to write make one thread responsible for writing them using information passed through the JMS subsystem.

Ultimately there's no paint by numbers way to unit test MT code to make sure its not vulnerable to a race condition.

Answer 6

Code that can possibly fail due to non-deterministic threading issues should not be tested. Instead, it should be deleted, and replaced with something that can be.

Most of the time this can be done simply by making everything single-threaded - only a small fraction of apps need the minor scalability boost that in-process pre-emptive scheduling provides. Alternatively, you can set up a threading architecture where the OS, compiler or a third-party tool is used to provide a hard guarantee that non-deterministic operations cannot take place.