如何使用ExecutorService递归调度任务

Question

我可以以某种方式在Java中使用ExecutorService来安排递归任务吗？

示例代码:(注意：为了增强可读性，省略了Thread.sleep的try / catch）

final ExecutorService executor = Executors.newFixedThreadPool(4);
for (int i = 0; i < 8; i++) {
    executor.execute(() -> { // TASK 1
        Thread.sleep(100); // simulate some work

        executor.execute(() -> { // TASK2
            Thread.sleep(500); // simulate some longer work
        });

    });
}

# terminate when all tasks proceeded
executor.shutdown();
executor.awaitTermination(9999999, TimeUnit.DAYS);

问题似乎是执行的顺序：

1. 创建执行者
2. 计划执行任务1（8x）
3. 关掉
4. awaitTermination
5. 第一次睡眠结束，然后安排执行任务2
6. 失败，因为关机后不会执行任务。

我相信这是常见的问题。 但是，我找不到任何合适的解决方案。 我寻求的清单：

要求：

• 能够从已安排的任务中执行任务
• 最后以某种方式阻止所有任务完成
• 没有主动等待任务执行
• 最大固定 使用的线程数
• 可能是无锁和线程安全的

请你帮助我好吗？ 我不想实现自己的线程池，因为它必须已经多次实现（最好是在标准库中）。

谢谢

Answer 1

您的核心问题是某些任务可能会或可能不会在运行期间产生更多任务。 因此，您必须至少等待该任务完成才能关闭。 你不能使用executor.shutdown()否则你肯定会提前关闭池。

您必须实现某种机制来编排哪些任务必须等待另一个任务完成，并且您必须维护一个在关闭池之前必须完成的所有任务的列表。

以下是您需要做的基本演示。 根据任务的相互关系要求，您可能需要更复杂的东西。

基本上，使用Callable和Future<Void>.get()等待必须完成的任何任务。

final ExecutorService executor = Executors.newFixedThreadPool(4);

class Task1 implements Callable<Void> {

    @Override
    public Void call() throws Exception {
        Thread.sleep(100); // simulate some work
        return null;
    }

}

class Task2 implements Callable<Void> {

    final Future<Void> waitFor;

    Task2(Future<Void> waitFor) {
        // This task must wait for a previous task to complete before commencement.
        this.waitFor = waitFor;
    }

    @Override
    public Void call() throws Exception {
        // Wait for the first task to complete.
        waitFor.get();
        Thread.sleep(100); // simulate some work
        return null;
    }

}

public void test() throws InterruptedException {
    // All of these tasks must complete before we close down the pool.
    List<Future<Void>> waitFor = new ArrayList<>();
    for (int i = 0; i < 8; i++) {
        Future<Void> f1 = executor.submit(new Task1());
        // We must wait for f1 to complete.
        waitFor.add(f1);
        // No need to wait for f2.
        executor.submit(new Task2(f1));
    }
    // Wait for all of the primary tasks to complete.
    for (Future<Void> wait : waitFor) {
        try {
            wait.get();
        } catch (InterruptedException | ExecutionException ex) {
            Logger.getLogger(Test.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
    // Can now shut down - will wait for all sub-tasks to complete because they are all in the queue now.
    executor.shutdown();
    executor.awaitTermination(9999999, TimeUnit.DAYS);
}

Answer 2

您可以尝试基于PriorityBlockingQueue创建ThreadPoolExecutor 。 然后你将shutDownTask放到Queue的末尾，它将等待其他人完成然后关闭池。 这是一个例子：

import java.util.concurrent.PriorityBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class TestExec {

    public static void main(String... s){

        PriorityBlockingQueue<Runnable> queue = new PriorityBlockingQueue<>(10,
                (o1,o2)-> {return o1 instanceof MyRunnable ? 1 : (o2 instanceof MyRunnable ? -1 : 0);} // shutDownTask at bottom of queue
                );
        ThreadPoolExecutor executor = new ThreadPoolExecutor(4, 4,
                                      0L, TimeUnit.MILLISECONDS,
                                      queue);

        //recursive tasks
        Runnable task3 = () -> {
            try {
                Thread.sleep(300);
            } catch (Exception e) { e.printStackTrace(); }
            System.out.println("done task 3");
        };
        Runnable task2 = () -> {
            try {
                Thread.sleep(200);
            } catch (Exception e) { e.printStackTrace(); }
            executor.execute(task3);
            System.out.println("done task 2");
        };
        Runnable task1 = () -> {
            try {
                Thread.sleep(100);
            } catch (Exception e) { e.printStackTrace(); }
            executor.execute(task2);
            System.out.println("done task 1");
        };

        for (int i = 0; i < 8; i++) {
            executor.execute(task1);
        }

        //shutDownTask
        MyRunnable r = ()->{
            while(executor.getActiveCount() > 1){ //wait until other tasks to be done
                try {
                    Thread.sleep(100);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            executor.shutdown();
            try {
                executor.awaitTermination(1, TimeUnit.SECONDS);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }};
        executor.execute(r);
    }

    private static interface MyRunnable extends Runnable{
        // dummy Runnable
    }
}

Answer 3

一个选项可以是使用afterExecute方法并更改shutdown方法的行为。 这是一个小型的可运行演示程序：

import java.util.concurrent.*;

public class TpShutdownAfterTasksDone extends ThreadPoolExecutor {

    private volatile boolean shutdown;
    private final CountDownLatch preShutdown = new CountDownLatch(1);

    public TpShutdownAfterTasksDone() {
        this(0, Integer.MAX_VALUE,
                60L, TimeUnit.SECONDS,
                new SynchronousQueue<Runnable>());
        // Copied from
        // Executors.newCachedThreadPool();
    }

    public TpShutdownAfterTasksDone(int corePoolSize, int maximumPoolSize,
            long keepAliveTime, TimeUnit unit,
            BlockingQueue<Runnable> workQueue) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
    }

    @Override
    public void shutdown() {

        shutdown = true;
        if (getActiveCount() < 1 && getQueue().size() < 1) {
            println("Immediate shutdown");
            realShutdown();
        }
    }

    protected void realShutdown() {

        println("Real shutdown");
        super.shutdown();
        preShutdown.countDown();
    }

    @Override
    protected void afterExecute(Runnable r, Throwable t) {

        super.afterExecute(r, t);
        if (shutdown && getActiveCount() <= 1 && getQueue().size() < 1) {
            // see http://stackoverflow.com/q/21277746/3080094
            // active count is still 1 when this method is executed after the last task.
            println("AfterExecute shutdown");
            realShutdown();
        } else {
            println("Tasks: " + getActiveCount());
        }
    }

    @Override
    public boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException {

        if (!preShutdown.await(timeout, unit)) {
            return false;
        }
        // TODO: calculate remaining timeout period
        return super.awaitTermination(timeout, unit);
    }

    public static void main(String[] args) {

        ThreadPoolExecutor tp = new TpShutdownAfterTasksDone();
        try {
            println("Executing test runnable.");
            tp.execute(new TestRunRunnable(tp));
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            println("Shutting tp down.");
            tp.shutdown();
            try { 
                println("Awaiting tp termination.");
                if (tp.awaitTermination(2L, TimeUnit.SECONDS)) {
                    println("Tp terminated.");
                } else {
                    throw new RuntimeException("Tp did not terminate.");
                }
            } catch (Exception e) {
                e.printStackTrace();
                tp.shutdownNow();
            }
        }

    }

    static class TestRunRunnable implements Runnable {

        ThreadPoolExecutor tp;
        public TestRunRunnable(ThreadPoolExecutor tp) {
            this.tp = tp;
        }
        @Override
        public void run() {

            sleep(100);
            tp.execute(new Runnable() {
                @Override
                public void run() {
                    sleep(500);
                }
            });
        }
    }

    private static void sleep(int time) {

        try {
            println("Sleeping " + time);
            Thread.sleep(time);
            println("Slept " + time);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    private static final long START_TIME = System.currentTimeMillis(); 

    private static void println(String msg) {
        System.out.println((System.currentTimeMillis() - START_TIME) + "\t " + msg);
    }

}

输出类似于：

2    Executing test runnable.
3    Shutting tp down.
3    Sleeping 100
3    Awaiting tp termination.
103  Slept 100
103  Tasks: 2
104  Sleeping 500
604  Slept 500
604  AfterExecute shutdown
604  Real shutdown
604  Tp terminated.