[英]Continuously submitting Runnable tasks to ExecutorService until work is done, getting java.util.concurrent.RejectedExecutionException
My multi-threaded class is supposed to carry out three operations – operation1
, operation2
, and operation3
– on a number of objects of the class ClassA
, where each type of operation is dependant on the earlier operation.我的多线程类应该对
ClassA
类的多个对象执行三个操作—— operation1
、 operation2
和operation3
,其中每种类型的操作都依赖于较早的操作。 For this, I have tried to implement the producer-consumer pattern using a number of BlockingQueue
s and an ExecutorService
.为此,我尝试使用许多
BlockingQueue
和一个ExecutorService
来实现生产者-消费者模式。
final ExecutorService executor = ForkJoinPool.commonPool();
final BlockingQueue<ClassA> operationOneQueue = new ArrayBlockingQueue<>(NO_OF_CLASS_A_OBJECTS);
final BlockingQueue<ClassA> operationTwoQueue = new ArrayBlockingQueue<>(NO_OF_CLASS_A_OBJECTS);
final BlockingQueue<ClassA> operationThreeQueue = new ArrayBlockingQueue<>(NO_OF_CLASS_A_OBJECTS);
final BlockingQueue<ClassA> resultQueue = new ArrayBlockingQueue<>(NO_OF_CLASS_A_OBJECTS);
The operations are implemented like this:操作是这样实现的:
void doOperationOne() throws InterruptedException {
ClassA objectA = operationOneQueue.take();
objectA.operationOne();
operationTwoQueue.put(objectA);
}
where each type of operation has its own corresponding method, with its "own" in-queue and out-queue.其中每种类型的操作都有自己对应的方法,有“自己的”入队和出队。 Each operation method calls the appropriate method on the
ClassA
object.每个操作方法调用
ClassA
对象上的适当方法。 The method doOperationThree
puts ClassA
objects in the resultQueue
, meaning they have been completely processed.方法
doOperationThree
将ClassA
对象放入resultQueue
,这意味着它们已被完全处理。
First, I fill the operationOneQueue
with all ClassA
objects that are to be operated on.首先,我用所有要操作的
ClassA
对象填充operationOneQueue
。 Then, I try to assign executable tasks to the ExecutorService
like this:然后,我尝试将可执行任务分配给
ExecutorService
如下所示:
while (resultQueue.size() < NO_OF_CLASS_A_OBJECTS) {
executor.execute(() -> {
try {
doOperationOne();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
executor.execute(() -> {
try {
doOperationTwo();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
executor.execute(() -> {
try {
doOperationThree();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
}
executor.shutdown();
Running my program, I get a java.util.concurrent.RejectedExecutionException
.运行我的程序,我得到一个
java.util.concurrent.RejectedExecutionException
。
Operation1: ClassA object 0
Operation2: ClassA object 0
Operation1: ClassA object 1
Operation3: ClassA object 0
....
Operation1: ClassA object 46
Operation2: ClassA object 45
Operation3: ClassA object 45
Exception in thread "main" java.util.concurrent.RejectedExecutionException: Queue capacity exceeded
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.growArray(ForkJoinPool.java:912)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.lockedPush(ForkJoinPool.java:867)
at java.base/java.util.concurrent.ForkJoinPool.externalPush(ForkJoinPool.java:1911)
at java.base/java.util.concurrent.ForkJoinPool.externalSubmit(ForkJoinPool.java:1930)
at java.base/java.util.concurrent.ForkJoinPool.execute(ForkJoinPool.java:2462)
at concurrent.operations.Program1.main(Program1.java:96)
What am I doing wrong?我究竟做错了什么? How can I achieve this without over-saturating the thread pool?
如何在不使线程池过度饱和的情况下实现这一目标?
Edit: Full disclosure – this is homework with some requirements.编辑:完全公开——这是有一些要求的作业。 1. I must use
ForkJoinPool.commonPool()
and must not set the number of threads myself, 2. I must use the consumer-producer pattern, and 3. I must not modify ClassA
. 1. 我必须使用
ForkJoinPool.commonPool()
并且ForkJoinPool.commonPool()
设置线程数, 2. 我必须使用消费者-生产者模式,以及 3. 我不能修改ClassA
。
I really like doing concurrent stuff, so I did try writing it.我真的很喜欢并发的东西,所以我确实尝试编写它。 I did use
CompletableFuture
which a) does run in the ForkJoinPool.commonPool
by default and b) makes the actual processing really easy:我确实使用了
CompletableFuture
,它 a) 默认在ForkJoinPool.commonPool
中运行,b) 使实际处理变得非常简单:
while (true) {
final ClassA nextOperation = queue.take();
CompletableFuture.runAsync(nextOperation::operationOne)
.thenRun(nextOperation::operationTwo)
.thenRun(nextOperation::operationThree)
.thenRun(() -> resultQueue.add(nextOperation));
}
This will take ClassA
objects from the queue and execute all their operations concurrently, but in order.这将从队列中取出
ClassA
对象并按顺序并发执行它们的所有操作。
You did leave out where the tasks are coming from, and whether you need the consumer to terminate.您确实忽略了任务的来源,以及是否需要消费者终止。 Generally you don't want to, and it does make matters a bit more complicated.
通常你不想这样做,这确实让事情变得更加复杂。
private static final int COUNT = 10;
private static final Random RANDOM = new Random();
public static void main(String[] args) throws ExecutionException, InterruptedException {
BlockingQueue<ClassA> runnables = new ArrayBlockingQueue<>(COUNT);
BlockingQueue<ClassA> finished = new ArrayBlockingQueue<>(COUNT);
// start producer
ExecutorService createTaskExecutor = Executors.newSingleThreadExecutor();
createTaskExecutor.submit(() -> fillQueue(runnables));
// wait for all consumer tasks to finish
while (finished.size() != COUNT) {
try {
// we need to poll instead of waiting forever
// because the last tasks might still be running
// while there are no others to add anymore
// so we need to check again if all have finished in the meantime
final ClassA nextOperation = runnables.poll(2, TimeUnit.SECONDS);
if (nextOperation != null) {
CompletableFuture.runAsync(nextOperation::operationOne)
.thenRun(nextOperation::operationTwo)
.thenRun(nextOperation::operationThree)
.thenRun(() -> finished.add(nextOperation));
}
} catch (InterruptedException e) {
System.err.println("exception while retrieving next operation");
// we will actually need to terminate now, or probably never will
throw e;
}
}
System.out.printf("finished tasks (%d):%n", finished.size());
for (ClassA classA : finished) {
System.out.printf("finished task %d%n", classA.designator);
}
createTaskExecutor.shutdown();
}
private static void fillQueue(BlockingQueue<ClassA> runnables) {
// start thread filling the queue at random
for (int i = 0; i < COUNT; i++) {
runnables.add(new ClassA(i));
try {
Thread.sleep(RANDOM.nextInt(1_000));
} catch (InterruptedException e) {
System.err.println("failed to add runnable");
}
}
}
Since you didn't provide ClassA
, I used this one.由于您没有提供
ClassA
,我使用了这个。 It contains an identifier so you can track which is running at what time.它包含一个标识符,因此您可以跟踪哪个正在运行。
class ClassA {
private static final Random RANDOM = new Random();
public final int designator;
public ClassA(int i) {
designator = i;
}
public void operationOne() {
System.out.printf("%d: operation 1%n", designator);
sleep();
}
public void operationTwo() {
System.out.printf("%d: operation 2%n", designator);
sleep();
}
public void operationThree() {
System.out.printf("%d: operation 3%n", designator);
sleep();
}
private static void sleep() {
try {
Thread.sleep(RANDOM.nextInt(5_000));
} catch (InterruptedException e) {
System.err.println("interrupted while executing task");
}
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.