如何使用ExecutorService遞歸調度任務

Question

我可以以某種方式在Java中使用ExecutorService來安排遞歸任務嗎？

示例代碼:(注意：為了增強可讀性，省略了Thread.sleep的try / catch）

final ExecutorService executor = Executors.newFixedThreadPool(4);
for (int i = 0; i < 8; i++) {
    executor.execute(() -> { // TASK 1
        Thread.sleep(100); // simulate some work

        executor.execute(() -> { // TASK2
            Thread.sleep(500); // simulate some longer work
        });

    });
}

# terminate when all tasks proceeded
executor.shutdown();
executor.awaitTermination(9999999, TimeUnit.DAYS);

問題似乎是執行的順序：

1. 創建執行者
2. 計划執行任務1（8x）
3. 關掉
4. awaitTermination
5. 第一次睡眠結束，然后安排執行任務2
6. 失敗，因為關機后不會執行任務。

我相信這是常見的問題。 但是，我找不到任何合適的解決方案。 我尋求的清單：

要求：

• 能夠從已安排的任務中執行任務
• 最后以某種方式阻止所有任務完成
• 沒有主動等待任務執行
• 最大固定 使用的線程數
• 可能是無鎖和線程安全的

請你幫助我好嗎？ 我不想實現自己的線程池，因為它必須已經多次實現（最好是在標准庫中）。

謝謝

Answer 1

您的核心問題是某些任務可能會或可能不會在運行期間產生更多任務。 因此，您必須至少等待該任務完成才能關閉。 你不能使用executor.shutdown()否則你肯定會提前關閉池。

您必須實現某種機制來編排哪些任務必須等待另一個任務完成，並且您必須維護一個在關閉池之前必須完成的所有任務的列表。

以下是您需要做的基本演示。 根據任務的相互關系要求，您可能需要更復雜的東西。

基本上，使用Callable和Future<Void>.get()等待必須完成的任何任務。

final ExecutorService executor = Executors.newFixedThreadPool(4);

class Task1 implements Callable<Void> {

    @Override
    public Void call() throws Exception {
        Thread.sleep(100); // simulate some work
        return null;
    }

}

class Task2 implements Callable<Void> {

    final Future<Void> waitFor;

    Task2(Future<Void> waitFor) {
        // This task must wait for a previous task to complete before commencement.
        this.waitFor = waitFor;
    }

    @Override
    public Void call() throws Exception {
        // Wait for the first task to complete.
        waitFor.get();
        Thread.sleep(100); // simulate some work
        return null;
    }

}

public void test() throws InterruptedException {
    // All of these tasks must complete before we close down the pool.
    List<Future<Void>> waitFor = new ArrayList<>();
    for (int i = 0; i < 8; i++) {
        Future<Void> f1 = executor.submit(new Task1());
        // We must wait for f1 to complete.
        waitFor.add(f1);
        // No need to wait for f2.
        executor.submit(new Task2(f1));
    }
    // Wait for all of the primary tasks to complete.
    for (Future<Void> wait : waitFor) {
        try {
            wait.get();
        } catch (InterruptedException | ExecutionException ex) {
            Logger.getLogger(Test.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
    // Can now shut down - will wait for all sub-tasks to complete because they are all in the queue now.
    executor.shutdown();
    executor.awaitTermination(9999999, TimeUnit.DAYS);
}

Answer 2

您可以嘗試基於PriorityBlockingQueue創建ThreadPoolExecutor 。 然后你將shutDownTask放到Queue的末尾，它將等待其他人完成然后關閉池。 這是一個例子：

import java.util.concurrent.PriorityBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class TestExec {

    public static void main(String... s){

        PriorityBlockingQueue<Runnable> queue = new PriorityBlockingQueue<>(10,
                (o1,o2)-> {return o1 instanceof MyRunnable ? 1 : (o2 instanceof MyRunnable ? -1 : 0);} // shutDownTask at bottom of queue
                );
        ThreadPoolExecutor executor = new ThreadPoolExecutor(4, 4,
                                      0L, TimeUnit.MILLISECONDS,
                                      queue);

        //recursive tasks
        Runnable task3 = () -> {
            try {
                Thread.sleep(300);
            } catch (Exception e) { e.printStackTrace(); }
            System.out.println("done task 3");
        };
        Runnable task2 = () -> {
            try {
                Thread.sleep(200);
            } catch (Exception e) { e.printStackTrace(); }
            executor.execute(task3);
            System.out.println("done task 2");
        };
        Runnable task1 = () -> {
            try {
                Thread.sleep(100);
            } catch (Exception e) { e.printStackTrace(); }
            executor.execute(task2);
            System.out.println("done task 1");
        };

        for (int i = 0; i < 8; i++) {
            executor.execute(task1);
        }

        //shutDownTask
        MyRunnable r = ()->{
            while(executor.getActiveCount() > 1){ //wait until other tasks to be done
                try {
                    Thread.sleep(100);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            executor.shutdown();
            try {
                executor.awaitTermination(1, TimeUnit.SECONDS);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }};
        executor.execute(r);
    }

    private static interface MyRunnable extends Runnable{
        // dummy Runnable
    }
}

Answer 3

一個選項可以是使用afterExecute方法並更改shutdown方法的行為。 這是一個小型的可運行演示程序：

import java.util.concurrent.*;

public class TpShutdownAfterTasksDone extends ThreadPoolExecutor {

    private volatile boolean shutdown;
    private final CountDownLatch preShutdown = new CountDownLatch(1);

    public TpShutdownAfterTasksDone() {
        this(0, Integer.MAX_VALUE,
                60L, TimeUnit.SECONDS,
                new SynchronousQueue<Runnable>());
        // Copied from
        // Executors.newCachedThreadPool();
    }

    public TpShutdownAfterTasksDone(int corePoolSize, int maximumPoolSize,
            long keepAliveTime, TimeUnit unit,
            BlockingQueue<Runnable> workQueue) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
    }

    @Override
    public void shutdown() {

        shutdown = true;
        if (getActiveCount() < 1 && getQueue().size() < 1) {
            println("Immediate shutdown");
            realShutdown();
        }
    }

    protected void realShutdown() {

        println("Real shutdown");
        super.shutdown();
        preShutdown.countDown();
    }

    @Override
    protected void afterExecute(Runnable r, Throwable t) {

        super.afterExecute(r, t);
        if (shutdown && getActiveCount() <= 1 && getQueue().size() < 1) {
            // see http://stackoverflow.com/q/21277746/3080094
            // active count is still 1 when this method is executed after the last task.
            println("AfterExecute shutdown");
            realShutdown();
        } else {
            println("Tasks: " + getActiveCount());
        }
    }

    @Override
    public boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException {

        if (!preShutdown.await(timeout, unit)) {
            return false;
        }
        // TODO: calculate remaining timeout period
        return super.awaitTermination(timeout, unit);
    }

    public static void main(String[] args) {

        ThreadPoolExecutor tp = new TpShutdownAfterTasksDone();
        try {
            println("Executing test runnable.");
            tp.execute(new TestRunRunnable(tp));
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            println("Shutting tp down.");
            tp.shutdown();
            try { 
                println("Awaiting tp termination.");
                if (tp.awaitTermination(2L, TimeUnit.SECONDS)) {
                    println("Tp terminated.");
                } else {
                    throw new RuntimeException("Tp did not terminate.");
                }
            } catch (Exception e) {
                e.printStackTrace();
                tp.shutdownNow();
            }
        }

    }

    static class TestRunRunnable implements Runnable {

        ThreadPoolExecutor tp;
        public TestRunRunnable(ThreadPoolExecutor tp) {
            this.tp = tp;
        }
        @Override
        public void run() {

            sleep(100);
            tp.execute(new Runnable() {
                @Override
                public void run() {
                    sleep(500);
                }
            });
        }
    }

    private static void sleep(int time) {

        try {
            println("Sleeping " + time);
            Thread.sleep(time);
            println("Slept " + time);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    private static final long START_TIME = System.currentTimeMillis(); 

    private static void println(String msg) {
        System.out.println((System.currentTimeMillis() - START_TIME) + "\t " + msg);
    }

}

輸出類似於：

2    Executing test runnable.
3    Shutting tp down.
3    Sleeping 100
3    Awaiting tp termination.
103  Slept 100
103  Tasks: 2
104  Sleeping 500
604  Slept 500
604  AfterExecute shutdown
604  Real shutdown
604  Tp terminated.