使用Stream API或CompletableFuture的并行数据库查询？

Question

Assume I want to send multiple database queries (or webservice requests) in parallel, and aggregate them afterwards. Would I better use the stream API or CompletableFuture ?

STREAM:

List<Result> result = requests.parallelStream()
                              .map(req -> query(req.getFirstname, req.getLastname))
                              .collect(toList());

//a database, or a webservice
private Result query(firstname, lastname);

FUTURES:

List<CompletableFuture> futures;
for (QueryReq req: requests) { //given input
    futures.add(CompletableFuture
                        .supplyAsync(() -> query(req.getFirstname, req.getLastname));
}

//wait for all futures to complete and collect the results
List<Result> results = new ArrayList<>();
for (CompleteableFuture f : futures) {
   results.add(f.get());
}

While stream is certainly less verbose, but which one should be preferred for what reasons?

Sidenote : I know I could query this example easier with sql = :firstname IN (..) and ... :lastname IN(..) . But it's just an example about wether to use stream or futures.

The task could as well be to send multiple webservice requests in parallel, instead of db queries.

Answer 1

As you already said: "stream is certainly less verbose", isn't it enough to prefer using Stream for you? To be fair, I think we also should rewrite the second sample code with CompletableFuture by Java 8 Stream APIs.

List<Result> result = requests.stream()
        .map(req -> CompletableFuture.supplyAsync(() -> query(req.getFirstname, req.getLastname)))
        .collect(toList()).stream()
        .map(f -> f.get()).collect(toList());

It looks it's still much verbose/longer than:

List<Result> result = requests.parallelStream()
        .map(req -> query(req.getFirstname, req.getLastname)).collect(toList());

However, I think here the key point is how to set the concurrent thread number: by parallel stream, the thread number is fixed by ForkJoinPool.commonPool , the CPU-core number. Usually that's too small for sending big amount web/db requests. For example, if there're tens/hundreds of web/db requests to send, Most times it's much faster to send the requests with 20 or more threads than the thread number defined in ForkJoinPool.commonPool . personally, I also don't know what's convenient way to specified the thread number in parallel stream. Here are some answers you can refer to: custom-thread-pool-in-java-8-parallel-stream