Huge delay in CompletableFuture join() method

Question

So I'm working on an application that has to make 20+ HTTP calls at a time. Each one of them takes 2-3 seconds to get a response. It's pretty slow to make these calls one at a time (40 seconds at best), so I am trying to send them asynchronously via CompletableFutures. This should allow me to make calls while I'm waiting for the response of others, in theory reducing the total time to maybe 4-5 seconds instead of 40.

I made a very similar setup to this tutorial I found at https://www.codepedia.org/ama/how-to-make-parallel-calls-in-java-with-completablefuture-example .

import org.codingpedia.example;

import javax.inject.Inject;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.function.Supplier;
import java.util.stream.Collectors;

public class ParallelCallsDemoService {

    @Inject
    RestApiClient restApiClient;

    private ExecutorService es = Executors.newFixedThreadPool(20);

    public List<ToDo> getToDos(List<String> ids){

        List<CompletableFuture<ToDo>> futures =
                ids.stream()
                          .map(id -> getToDoAsync(id))
                          .collect(Collectors.toList());

        List<ToDo> result =
                futures.stream()
                        .map(CompletableFuture::join)
                        .collect(Collectors.toList());

        return result;
    }


    CompletableFuture<ToDo> getToDoAsync(String id){

        CompletableFuture<ToDo> future = CompletableFuture.supplyAsync(() -> {
            return restApiClient.makeSomeHttpCall(id);
        }, es);

        return future;
    }

}

By all accounts it seems to be working - the calls all get sent at roughly the same time, and they all return in a couple seconds. But then I'm experiencing a huge delay of 30-40 seconds on this part:

        List<ToDo> result =
                futures.stream()
                        .map(CompletableFuture::join)
                        .collect(Collectors.toList());

This makes it take roughly the same time as sending serially, which baffles me. How can it be that I'm getting all responses in a couple seconds but then there's a 30 second delay on joining them? It's almost as if (despite appearances) they're still being made serially. Why does the join take so long?

Answer 1

There's a bit of a problem here

List<ToDo> result =
                futures.stream()
                        .map(CompletableFuture::join)
                        .collect(Collectors.toList());

I think the stream you are using is not a parallel stream. Therefore, each call to map is waiting for the last call to finish. Changing futures.stream() to futures.parallelStream() should have an improvement. Of course if you are not using a machine with a single core.

Answer 2

Finally figured this out! Thanks everyone for your suggestions. Turns out it was nothing to do with my implementation of CompletableFutures. When I receive a response from the service, I use JAXB to convert the java object to an XML string for logging purposes. I started looking at thread dumps when it was hanging, and realized it was actually the JAXB string conversion that the threads were waiting for (the response object is quite large). I took out that part, and the performance immediately improved to what it should have been.

Answer 3

we have encountered a similar problem. we have solved it using '.get(timeout)' method of CompletableFuture.

CompletableFuture[] array = (CompletableFuture[]) futures.toArray();
       try {
        CompletableFuture.allOf(array).get(180, TimeUnit.SECONDS);
    } catch (InterruptedException | ExecutionException | TimeoutException e) {
        //log error
    }

Place the timeout based on your real-time results. you can tweak the time using an external configuration.