使用 ExecutorService 并行化和监控任务

Question

I have a function/task void task() that needs to be called about 4-6 million times.
I want to parallelize this operation over threads in a thread pool.
I do not care about the return value of the tasks so I can avoid messing around with Future<T> .
I want to periodically poll the status of how the threads are coming along. The status is simply how many invocations of task() returned cleanly and how many threw an exception.

Here's what I came up with:

class Test {
    AtomicInteger success = new AtomicInteger(0);
    AtomicInteger failed = new AtomicInteger(0);
    CountDownLatch latch = new CountDownLatch(1_000_000);

    private void start() {
        ExecutorService executorService = Executors.newFixedThreadPool();
        for (int i = 0; i < 1_000_000; i++) {
          executorService.execute(this::task);
        }
        while (!countDownLatch.await(1, TimeUnit.SECONDS)) {
          log("Success: %d Failed: %d", success.get(), failed.get());
        }
        log("===================== Final tally =====================");
        log("Success: %d Failed: %d", success.get(), failed.get());
        executorService.shutdown();
    }

    private void task() {
        try {
           doSomeStuff();
           success.incrementAndGet()
        } catch(Exception e) {
           failed.incrementAndGet();
        }
        countDownLatch.countDown();
    }
}

Two AtomicInteger that the threads use to record successes or failures and a CountDownLatch that the "monitor" thread uses to check on the progress.

Is there a more idiomatic way to do this? Some thing that doesn't involve submitting millions of Runnable lambdas to the ExecutorService perhaps?

I could put the whole thing in an

IntStream.range(0, 1_000_000).parallelStream().map(...).groupBy(...)

but I won't be able to monitor the progress.

Answer 1

If you want to use stream as you proposed, you can move your monitoring part into another thread like this:

new Thread(()->{  
 while (!countDownLatch.await(1, TimeUnit.SECONDS)) {
          log("Success: %d Failed: %d", success.get(), failed.get());
        }
        log("===================== Final tally =====================");
        log("Success: %d Failed: %d", success.get(), failed.get());
}).start()

IntStream.range(0, 1_000_000).parallelStream().map(...).groupBy(...)