Asynchronous process on Executor Service

Question

I am using executor service for multi threading. Spanned across 15 threads, current process does completes 15 threads before moving on to next executor services which is again spanned across 15 threads. I am looking for code help where if any of current threads in step 1 executor services completes processing, i need to move on to next executor services which can start using the threads instead of waiting on completion of 15 threads in step 1 to complete.

I want to utilize the threads completed and move on to step 2 even if other threads are getting executed in step1 and as soon as each thread in step 1 gets completed, step 2 can grab and continue processing. Suggestions pls

// Step 1 
ExecutorService executor1 = Executors.newFixedThreadPool(15);
for (int i=0;i<=15;i++) { 
    Runnable worker = new Runnable("Step 1 Insert"); 
    executor1.execute(worker); } 
executor1.shutdown(); 
// Step 2 
ExecutorService executor2 = Executors.newFixedThreadPool(15);
for (int i=0;i<=15;i++) { 
    Runnable worker = new Runnable("Step 2 Insert"); 
    executor2.execute(worker); } 
executor2.shutdown();

Answer 1

You can use ExecutorCompletionService . I slightly modified example from JavaDoc

ExecutorService executor1 = Executors.newFixedThreadPool(15);
ExecutorService executor2 = Executors.newFixedThreadPool(15);

....

CompletionService<Result> ecs = new ExecutorCompletionService<Result>(executor1);
for (Callable<Result> s : solvers)
    ecs.submit(s);
int n = solvers.size();
for (int i = 0; i < n; ++i) {
    Result r = ecs.take().get();
    if (r != null)
        executor2.submit(r);
}
executor1.shutdown();
//and shutdown executor2 when you don't need it

Result class is just for example and I assume it implements Callable or Runnable so it could be submitted into the second executor.

You can also init instance of ExecutorService as new ThreadPoolExecutor(0, 15, 10, TimeUnit.SECONDS, new LinkedBlockingQueue()) to terminate threads which are not needed right now, but you want to reuse this executor later.

Or you can simply use one executor. After receiving a completed task from ExecutorCompletionService resubmit it (or its result) into the same executor (Thanks @Fildor). But in this case you need to determine when task is completed first time.

for (int i = 0; i < n;) {
    Result r = ecs.take().get();
    if (r != null && isCompletedFirstTime(r))//some user defined function
        executor1.submit(r);
    else 
        ++i;//we need to know when to stop, otherwise we'll stuck in `take()`
}

Answer 2

Why can't you just do all the steps in the same Runnable? eg

ExecutorService executor = Executors.newFixedThreadPool(15);
for (int i=0;i<=15;i++) { 
    Runnable worker = new Runnable() {
        public void run() {
            doStep1();
            doStep2();
            doStep3();
            ...
        }
    }; 
    executor.execute(worker);
}
executor.shutdown();

Answer 3

@SpiderPig's answer is one good solution IMHO, yet I would like to give an alternative in case you want to decouple steps:

Use one single Executor, in your case the requirement seems to be a FixedThreadPool with 15 Threads.

Next would be to define Step-Runnables like so:

class StepX implements Runnable{
    private final State _state; // Reference to the data to work on.
    StepX( State state ){
        _state = state;
    }

    public void run(){ 
        // work on _state
        executor.submit( new StepXplusOne( _state ) ); // reference executor in outer class and schedule next step.
    }
} 

You can see that I used a State object, which holds all data you need to perform the steps and collect the result. Of course you'd need to define StepX as Step1, Step2, ...

In the outer class, you'd only have to submit N Step1-Runnables and it will only use your 15 Threads and go through the steps.

I left out a means to signal when all steps are done, because there are plenty of possibilities to do this and I am sure you can pick one by yourself.

Answer 4

You can try use CountDownLatch class.

http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/CountDownLatch.html

    final CountDownLatch latch = new CountDownLatch(1);

    ExecutorService executor1 = Executors.newFixedThreadPool(15);
    for (int i = 0; i <= 15; i++) {
        Runnable worker = new Runnable() {

            @Override
            public void run() {
                latch.countDown();

            }
        };
        executor1.execute(worker);
    }

    latch.await();

    ExecutorService executor2 = Executors.newFixedThreadPool(15);
    for (int i = 0; i <= 15; i++) {
        Runnable worker = new Runnable("Step 2 Insert");
        executor2.execute(worker);
    }
    executor2.shutdown();