Java中如何實現線程安全的ExecutorService？

Question

我正在自己實施 ExecutorService 以了解 Java 內部結構。 雖然 ExecutorService 在單個線程中調用時可以工作，但我不確定如何實現它以便多個線程可以同時將任務提交給同一個執行器服務。 如何在現有代碼中實現此功能？

我寫的代碼如下。

import java.util.ArrayList;
import java.util.Collection;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.*;

public class MyExecutorService implements ExecutorService {
    private final MyWorkerThread[] workerThreads;
    private final LinkedList<MyTask> taskQueue;
    private final LinkedList<Object> completed;
    private boolean terminated;
    private boolean hasBeenShutDown;
    private final Object shutDownLock;

    public MyExecutorService(int numThreads) {
        workerThreads = new MyWorkerThread[numThreads];
        completed = new LinkedList<>();
        taskQueue = new LinkedList<>();
        shutDownLock = new Object();
        for (int i = 0; i < numThreads; i++) {
            workerThreads[i] = new MyWorkerThread();
            workerThreads[i].start();
        }
        terminated = false;
        hasBeenShutDown = false;
    }

    @Override
    public void shutdown() {
        hasBeenShutDown = true;
        for (int i = 0; i < workerThreads.length; i++) {
            submitToStop();
        }
        for (MyWorkerThread worker : workerThreads) {
            try {
                worker.join();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
        synchronized (shutDownLock) {
            shutDownLock.notifyAll();
        }
    }

    @Override
    public List<Runnable> shutdownNow() {
        hasBeenShutDown = true;
        for (MyWorkerThread workerThread : workerThreads) {
            workerThread.interrupt();
        }
        LinkedList<Runnable> pendingTasks = new LinkedList<>();
        for (MyTask task : taskQueue) {
            pendingTasks.add(task.getFutureTask());
        }
        return pendingTasks;
    }

    @Override
    public boolean isTerminated() {
        return terminated;
    }

    @Override
    public boolean isShutdown() {
        return hasBeenShutDown;
    }


    @Override
    public boolean awaitTermination(long timeout, TimeUnit unit) {
        System.out.println("Awaiting Termination");
        synchronized (shutDownLock) {
            try {
                shutDownLock.wait(unit.convert(timeout, unit));
                terminated = true;
            } catch (InterruptedException e) {
                // ignore
            }
        }
        return hasBeenShutDown;
    }

    @Override
    public <T> Future<T> submit(Callable<T> task) {
        FutureTask<T> futureTask = new FutureTask<>(task);
        synchronized (taskQueue) {
            taskQueue.add(new MyTask(false, futureTask));
            taskQueue.notifyAll();
        }
        return futureTask;
    }

    @Override
    public <T> Future<T> submit(Runnable task, T result) {
        FutureTask<T> futureTask = new FutureTask<>(task, result);
        synchronized (taskQueue) {
            taskQueue.add(new MyTask(false, futureTask));
            taskQueue.notifyAll();
        }
        return futureTask;
    }

    @Override
    public Future<?> submit(Runnable task) {
        FutureTask<?> futureTask = new FutureTask<>(task, "RESULT");
        synchronized (taskQueue) {
            taskQueue.add(new MyTask(false, futureTask));
            taskQueue.notifyAll();
        }
        return futureTask;
    }

    @Override
    public <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks) {
        List<Future<T>> futures = new ArrayList<>();
        for (Callable<T> task : tasks) {
            futures.add(submit(task));
        }
        return futures;
    }

    @Override
    public <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks, long timeout, TimeUnit unit) {
        FutureTask<List<Future<T>>> futureWorkerTask = new FutureTask<>(() -> invokeAll(tasks));
        try {
            return futureWorkerTask.get(timeout, unit);
        } catch (InterruptedException | ExecutionException | TimeoutException e) {
            return new ArrayList<>();
        }
    }

    @Override
    public <T> T invokeAny(Collection<? extends Callable<T>> tasks) throws InterruptedException {
        invokeAll(tasks);
        synchronized (completed) {
            while (completed.isEmpty()) {
//                System.out.println("Waiting");
                completed.wait();
            }
        }
        return (T) completed.removeFirst();
    }

    @Override
    public <T> T invokeAny(Collection<? extends Callable<T>> tasks, long timeout, TimeUnit unit) {
        FutureTask<T> futureWorkerTask = new FutureTask<>(() -> invokeAny(tasks));
        try {
            return (T) futureWorkerTask.get(timeout, unit);
        } catch (InterruptedException | ExecutionException | TimeoutException e) {
            return null;
        }
    }

    @Override
    public void execute(Runnable task) {
        FutureTask<String> futureTask = new FutureTask<>(task, "RESULT");
        synchronized (taskQueue) {
            taskQueue.add(new MyTask(false, futureTask));
            taskQueue.notifyAll();
        }
    }

    private void submitToStop() {
        synchronized (taskQueue) {
//            System.out.println("added to queue");
            taskQueue.add(new MyTask(true, null));
            taskQueue.notify();
        }
    }

    private class MyWorkerThread extends Thread {
        public void run() {
            while (true) {
                MyTask myTask;
                try {
                    synchronized (taskQueue) {
                        while (taskQueue.isEmpty())
                            taskQueue.wait();
                        myTask = taskQueue.removeFirst();
                    }
                    if (myTask.toStop)
                        break;
                    FutureTask futureTask = myTask.getFutureTask();
                    futureTask.run();
//                    System.out.println("Running");
                    synchronized (completed) {
                        completed.add(futureTask.get());
                        completed.notifyAll();
                    }
                } catch (Exception e) {
                    break;
                }
            }
        }
    }

    private static class MyTask {
        private FutureTask task;
        private final boolean toStop;

        MyTask(boolean toStop, FutureTask task) {
            this.task = task;
            this.toStop = toStop;
        }

        public FutureTask getFutureTask() {
            return task;
        }
    }
}

Answer 1

你有你的同步隊列，它應該可以工作（我只是在看你的提交）。 不過，只需徹底驗證這一點，為每個公共方法編寫一些適當的多線程測試。 總是有 ConcurrentLinkedQueue 而不是 LinkedList，它將在內部為您處理同步方面； 看看這里如何使用 ConcurrentLinkedQueue？

在回答您的評論時編輯 invokeAny() 方法：

所以在 API 描述中，它說它應該

/**
 * Executes the given tasks, returning the result
 * of one that has completed successfully (i.e., without throwing
 * an exception), if any do. Upon normal or exceptional return,
 * tasks that have not completed are cancelled.
 * The results of this method are undefined if the given
 * collection is modified while this operation is in progress.
 *
 * @param tasks the collection of tasks
 * @param <T> the type of the values returned from the tasks
 * @return the result returned by one of the tasks
 * @throws InterruptedException if interrupted while waiting
 * @throws NullPointerException if tasks or any element task
 *         subject to execution is {@code null}
 * @throws IllegalArgumentException if tasks is empty
 * @throws ExecutionException if no task successfully completes
 * @throws RejectedExecutionException if tasks cannot be scheduled
 *         for execution
 */
<T> T invokeAny(Collection<? extends Callable<T>> tasks)
    throws InterruptedException, ExecutionException;

並查看您的代碼，這似乎就是您正在做的事情。

然后我只是偷看了java自己的“主要”基本實現，

java.util.concurrent.AbstractExecutorService

當然，它確實更多（我猜你的 IDE 和我的 Intellij 一樣，包括一個反編譯器——你自己看看吧）。

由於 Java 本身帶有 ExecutorService 的“工業實力”變體，因此自己構建一個並不常見。 您自定義的內容是“一級”，工作線程應該做什么，努力提供一些任務ID處理方式，在各種異常、超時等情況下該怎么做，等等，編寫自己的 ExecutorService 是不尋常的，但看看 Java 本身是如何實現這些東西的肯定是一個很好的練習，你會看到在編寫並發代碼時應該始終特別考慮並發的哪些方面。