Can I have blocking remote call in Flux.generate state generator

Question

I am generating a Flux from a series of blocking REST API calls, each call depends on the result of previous call. ie

Result r1 = call(fromRow = 0);
Result r2 = call(fromRow = 0 + r1.size());
Result r3 = call(fromRow = 0 + r1.size() + r2.size());
...

Here is a simplified version of what I am trying with:

Flux.generate(() -> 0, (i, sink) -> {
    Result r = slowRemoteCall(i);
    if (r == null) {
        sink.complete();
    } else {
        sink.next(r)
    }
    return i + r.size();
}, state -> {});

Just wonder will the blocking call slowRemoteCall inside the state generator become a problem?

Thank you for your help in advance!

Answer 1

With expand operator and a reactive remote client (eg: Spring WebClient) you can implement this in a reactive non-blocking way:

slowRemoteCall(0)
        .expand(result -> {
            if (result.size() == 0) { // stop condition
                return Mono.empty();
            } else {
                return slowRemoteCall(result.startIndex() + result.size()); // maintain state
            }
        })

Mono<Result> slowRemoteCall(int startIndex) {
    // simulate latency (could be a WebClient call here)
    return Mono.delay(Duration.ofMillis(200)).thenReturn(new Result(startIndex));
}

Inspired by this blog post .

Answer 2

It may become a problem! If you have any blocking call, you can use schedulers to get the task done by specific thread pools.

Flux.generate(.........)
    .subscribeOn(Schedulers.boundedElastic())

More info:

https://www.vinsguru.com/reactive-programming-schedulers/