Java - Reusing Thread Pool

Question

For my Java project I want to execute two tasks in parallel and combine their results. The problem is that I have to do this in a method that gets invoked many times per execution (maybe 1000 or more). I tried with ExecutorService and its thread pool, but if I declare the ExecutorService as a class field and begin submitting work to it, at some moment it throws a

java.lang.OutOfMemoryError: unable to create new native thread

So I decided to go with this code:

ExecutorService executor = Executors.newFixedThreadPool(2); 

Future<Integer> taskOne = executor.submit(new Callable<Integer> () {
    public Integer call() throws Exception {
        //Do work...
        return 0;
    }
});

Future<Integer> task2 = executor.submit(new Callable<Integer> () {
    public Integer call() throws Exception {
        //Do work...
        return 0;
    }
});

task1.get();
task2.get();

executor.shutdown();

while(!executor.isTerminated());

The problem is that with this new approach, the application performance is even worse compared to the single threaded solution. I think it's due to the overhead of creating a new thread pool on each method invocation.

So, my question is: is there a way of doing this same task but without having to create and shutdown the thread pool each method invocation?

Thank you.

Answer 1

I wouldn't necessarily say its an inferior design per se, but a few things to consider. Personally, I like thread pooling, and when used correctly, it's a robust solution.

The reason your performance decreased is that with the fixed pool size of 2, only two threads can ever be running at once. Chances are that you need a larger number - could be 10, could be 100 - but at least you can frame it such that you don't run out of memory.

That said, if the original design ran out of memory because too many threads were created, you definitely need to restrict the number of threads active at once, so your performance is going to take some hit...it just doesn't need to be all or nothing.

You could also use a limitedless executor thread pool, assuming that the running threads do return in a timely manner such that you still don't max out the threads. If the threads return in a somewhat deterministic fashion, when a thread is returned to the pool it's immediately available for the next available task without having to create something. So this is good.

That all said, looking at the exception message, it's also possible that at an operating system level you can't create more threads, regardless of what you've done within Java. I had a service running on CentOS and I legitimately had to increase the number of allowable threads per process (in this case, one Java process running my service) because there was a max and I was hitting it. If you're running some linux flavor, that could be it, too, especially since it looks like threads themselves are doing nothing but returning.

Answer 2

The class field solution that you described sounds like a better design. What was the size of that pool? You should debug the OutOfMemoryError instead of switching to a inferior design.