如何等到多个 WebClient Flux 请求完成？

Question

I want to:

1. subscribe to multiple endpoints returning Flux and output the messages I receive.
2. wait until all messages have been output from all endpoints before continuing.
3. avoid processing messages from multiple endpoints "together" (eg Flux.zip) because the endpoints will return an uneven number of messages and are not logically connected to each other
4. block forever if one or more endpoints generate an infinite number of messages

The following code satisfies 1 and 3 but not 2 and 4:

        Stream.of("http://service1.com", "http://service2.com", "http://service3.com")
                .forEach(service -> {
                    webClient.get()
                            .retrieve()
                            .bodyToFlux(String.class)
                            .map(message -> service + ": " + message)
                            .subscribe(System.out::println);
                });

        System.out.println("Received all messages");

The line "Received all messages" should not be printed until all endpoints have finished. However, because subscribe is asynchronous, that line is instead printed almost immediately and my application continues instead of waiting.

What should I do differently?

Answer 1

I believe the following code snippet achieves 3 out of 4 points in your question though I do not feel like I fully understand the 3rd requirement. Let me if this example meets what is needed and if not, what is missing.

        Flux.just("http://service1.com", "http://service2.com", "http://service3.com")
            .flatMap(url -> webClient.get()
                    .uri(url)
                    .retrieve()
                    .bodyToFlux(String.class)
                    .map(body -> url + ":" + body)
            )
            .collectList()
            .doOnNext(list -> LOG.info("Received all messages"))
            .subscribe(list -> LOG.info("" + list));

Answer 2

flatMap is one way to merge fluxes together but you can also use Flux.merge

    List<Flux<String>> individualResults =
        Stream.of("http://service1.com", "http://service2.com", "http://service3.com")
            .map(
                service ->
                    webClient //
                        .get()
                        .retrieve()
                        .bodyToFlux(String.class))
            .collect(toList());
    Flux<String> mergedResults = Flux.merge(individualResults);  // Will not complete until all individual Fluxes have completed.
    mergedResults //
        .doOnNext(System.out::println)
        .then()
        .block(); // block this thread until mergedResults completes
    System.out.println("Received all messages");