[英]How to return a Reactive Flux that contains a Reactive Mono and Flux?
[英]How to reassign a variable in reactive Flux stream?
我正在向 Web 服務發送請求,將結果轉換為大型csv
並將 csv 行保存到數據庫中。
由於請求長時間運行(10-20 秒),我想並行化請求。 我在單個StringBuilder
中收集所有數據,該 StringBuilder 包含轉換后的 csv 行。
問題:如果在 csv 中達到了我的 1000 行塊,我怎樣才能將數據取出以進行持久化,而任何其他並發響應都將寫入新的StringBuilder
?
因為,無法重新初始化 stream 的最終變量。
final StringBuilder sb = new StringBuilder();
AtomicInteger count = new AtomicInteger();
Flux.fromIterable(requests)
.flatMap(req -> {
return webClientService.send(req); //assume long running response
}, 8) //send 8 requests in parallel, as response takes up to 10s
.map(rsp -> {
//convert response to csv values and add to StringBuilder
int c = addCsv(sb, rsp);
if (count.addAndGet(c) > 1000) {
//TODO how can I assign a new StringBuilder,
//so that all further finished responses will append the csv to the new builder?
//same problem with the counter.
databaseWriter.write(sb.build()); //writes the content so far to db, but not threadsafe so far
}
return c;
})
.blockLast();
也許您可以嘗試完全避免副作用,例如:
.map(x -> toCsv(x))
.reduce((a, b) -> {
if (length(a) < 1000) {
return concat(a, b);
}
databaseWriter.write(a);
return b;
})
.doOnNext(x -> databaseWriter.write(x))
在我看來,您可以使用內置運算符來實現相同的結果:
Flux.fromIterable(requests)
.flatMap(req -> webClientService
.send(req)
.subscribeOn(Schedulers.boundedElastic()), 8)// subscribeOn to subscribe from different threads
.map(resp -> converToCsvLine(resp)) //make some transformations on the respnse
.window(1000) //split incoming data into 1000 lines
.flatMap(stringFlux -> stringFlux.collect(Collectors.joining("\n")))// collect last 1000
.flatMap(s -> Mono.fromRunnable(() -> writeToDb(s))) //do some logic on the collected 1000 lines
.blockLast();
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.