CompleteableFuture Java 8 异常行为

Question

I noticed some unusual behavior with CompleteableFutures in Java 8 with streaming.

String [] arr = new String[]{"abc", "def", "cde", "ghj"};
        ExecutorService executorService = Executors.newFixedThreadPool(10);
        List<String> lst =
                Arrays.stream(arr)
                        .map(r ->
                                CompletableFuture.supplyAsync(() -> {
                                    try {
                                        Thread.sleep(5000);
                                        return "e";
                                    } catch (Exception e) {
                                        e.printStackTrace();
                                    }
                                    return null;
                                }, executorService)
                        )
                        .map(CompletableFuture::join)
                        .collect(Collectors.toList());

This code above takes 4*5000 = 20 seconds to execute, so this means the futures are waiting on one another.

 String [] arr = new String[]{"abc", "def", "cde", "ghj"};
        ExecutorService executorService = Executors.newFixedThreadPool(10);
        List<CompletableFuture<String>> lst =
                Arrays.stream(arr)
                        .map(r ->
                                CompletableFuture.supplyAsync(() -> {
                                    try {
                                        Thread.sleep(5000);
                                        return "d";
                                    } catch (Exception e) {
                                        e.printStackTrace();
                                    }
                                    return null;
                                }, executorService)
                        )
                        .collect(Collectors.toList());

        List<String> s =
                lst
                        .stream()
                        .map(CompletableFuture::join)
                        .collect(Collectors.toList());

        System.out.println(s);

This code however, runs in 5 seconds meaning futures are running in parallel.

What I don't understand: In the second example I get a list of futures explicitly, then do a join, which takes 5 seconds, the first example I keep it streaming through and it seems to wait.

What's the reasoning behind this?

Answer 1

Streams don't necessarily do one stage, then the next. They can compose operations in any order they choose.

So for example,

Arrays.stream(array).map(e -> f(e)).map(e -> g(e)).collect(toList());

can end up being run the same way as

Arrays.stream(array).map(e -> g(f(e))).collect(toList());

...which would have the results you see: the futures are generated one at a time and immediately joined, instead of all being generated up front and then joined.

In point of fact, if you're not doing something async, it's usually more efficient to do it the second way. That way, the stream framework doesn't have to store all the results of f, then store all the results of g: it can only store the results of g(f(e)). The stream framework can't know you're doing async code, so it does the normal efficient thing.

Answer 2

I think the issue is with the second map function call in the original snippet. The map function is serial and hence calls the CF blocking function join for each one of the elements in the source array one after another.