[英]Spring WebFlux | How to wait till a list of Monos finish execution in parallel
I want to send n-number of requests to a REST endpoint in parallel.I want to make sure these get executed in different threads for performance and need to wait till all n requests finish.我想将 n 个请求并行发送到 REST 端点。我想确保这些请求在不同的线程中执行以获得性能并且需要等到所有 n 个请求完成。
Only way I could come up with is using CountDownLatch as follows (please check the main() method. This is testable code):我能想到的唯一方法是按如下方式使用 CountDownLatch(请检查 main() 方法。这是可测试的代码):
public static void main(String args[]) throws Exception {
int n = 10; //n is dynamic during runtime
final CountDownLatch waitForNRequests = new CountDownLatch(n);
//send n requests
for (int i =0;i<n;i++) {
var r = testRestCall(""+i);
r.publishOn(Schedulers.parallel()).subscribe(res -> {
System.out.println(">>>>>>> Thread: " + Thread.currentThread().getName() + " response:" +res.getBody());
waitForNRequests.countDown();
});
}
waitForNRequests.await(); //wait till all n requests finish before goto the next line
System.out.println("All n requests finished");
Thread.sleep(10000);
}
public static Mono<ResponseEntity<Map>> testRestCall(String id) {
WebClient client = WebClient.create("https://reqres.in/api");
JSONObject request = new JSONObject();
request.put("name", "user"+ id);
request.put("job", "leader");
var res = client.post().uri("/users")
.contentType(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromValue(request.toString()))
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.toEntity(Map.class)
.onErrorReturn(ResponseEntity.status(HttpStatus.SERVICE_UNAVAILABLE).build());
return res;
}
This doesnt look good and I am sure there is an elegant solution without using Latches..etc I tried following method,but I dont know how to resolve following issues :这看起来不太好,我确信有一个不使用闩锁的优雅解决方案..等我尝试了以下方法,但我不知道如何解决以下问题:
List<Mono<ResponseEntity<Map>>> lst = new ArrayList<>();
int n = 10; //n is dynamic during runtime
for (int i =0;i<n;i++) {
var r = testRestCall(""+i);
lst.add(r);
}
var t= Flux.fromIterable(lst).flatMap(Function.identity()); //tried merge() contact() as well
t.publishOn(Schedulers.parallel()).subscribe(res -> {
System.out.println(">>>>>>> Thread: " + Thread.currentThread().getName() + " response:" +res.getBody());
///??? all requests execute in a single thread.How to parallelize ?
});
//???How to wait till all n requests finish before goto the next line
System.out.println("All n requests finished");```
In reactive you think not about threads but about concurrency.在反应式中,你考虑的不是线程,而是并发。
Reactor executes non-blocking/async tasks on a small number of threads using Schedulers
abstraction to execute tasks. Reactor 使用
Schedulers
抽象来执行任务,在少量线程上执行非阻塞/异步任务。 Schedulers
have responsibilities very similar to ExecutorService
. Schedulers
的职责与ExecutorService
非常相似。 By default, for parallel scheduler number of threads is equal to number of CPU cores, but could be controlled by `reactor.schedulers.defaultPoolSize' system property.默认情况下,对于并行调度程序,线程数等于 CPU 内核数,但可以通过“reactor.schedulers.defaultPoolSize”系统属性进行控制。
In your example instead of creating multiple Mono
and then merge them, better to use Flux
and then process elements in parallel controlling concurrency.在您的示例中,与其创建多个
Mono
然后合并它们,不如使用Flux
然后以并行控制并发方式处理元素。
Flux.range(1, 10)
.flatMap(this::testRestCall)
By default, flatMap will process Queues.SMALL_BUFFER_SIZE = 256
number of in-flight inner sequences.默认情况下,flatMap 将处理
Queues.SMALL_BUFFER_SIZE = 256
个飞行中的内部序列。
You could control concurrency flatMap(item -> process(item), concurrency)
or use concatMap
operator if you want to process sequentially.您可以控制并发
flatMap(item -> process(item), concurrency)
或使用concatMap
运算符,如果你想按顺序处理。 Check flatMap(..., int concurrency, int prefetch) for details.检查flatMap(..., int concurrency, int prefetch)了解详情。
Flux.range(1, 10)
.flatMap(i -> testRestCall(i), 5)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.