[英]Linearization in Reactor Netty (Spring Boot Webflux)
How can I guarantee linearizability of requests in Reactor Netty?如何保证 Reactor Netty 中请求的线性化?
Theory:理论:
Given:鉴于:
Request A wants to write x=2, y=0请求A要写x=2,y=0
Request B wants to read x, y and write x=x+2, y=y+1请求B想读x,y,写x=x+2,y=y+1
Request C wants to read x and write y=x请求C想读x写y=x
All Requests are processed asynchronously and return to the client immediately with status ACCEPTED.所有请求都被异步处理,并立即以已接受状态返回给客户端。
Example:例子:
Send requests A, B, C in order.依次发送请求A、B、C。
Example Log Output: (request, thread name, x, y)示例日志 Output:(请求、线程名称、x、y)
Request A, nioEventLoopGroup-2-0, x=2, y=0请求 A,nioEventLoopGroup-2-0,x=2,y=0
Request C, nioEventLoopGroup-2-2, x=2, y=2请求 C,nioEventLoopGroup-2-2,x=2,y=2
Request B, nioEventLoopGroup-2-1, x=4, y=3请求 B,nioEventLoopGroup-2-1,x=4,y=3
Business logic requires all reads after A to see x=2 and y=0.业务逻辑要求 A 之后的所有读取都看到 x=2 和 y=0。
And request B to see x=2, y=0 and set y=1.并要求 B 查看 x=2、y=0 并设置 y=1。
And request C to see x=4 and set y=4.并请求 C 查看 x=4 并设置 y=4。
In short: The business logic makes every next write operation dependent on the previous write operation to be completed.简而言之:业务逻辑使每个下一个写操作都依赖于上一个要完成的写操作。 Otherwise the operations are not reversible.
否则操作不可逆。
Example Code示例代码
Document:文档:
@Document
@Data
@NoArgsConstructor
@AllArgsConstructor
public class Event {
@Id
private String id;
private int data;
public Event withNewId() {
setId(UUID.randomUUID().toString());
return this;
}
}
Repo:回购:
public interface EventRepository extends ReactiveMongoRepository<Event, String> {}
Controller: Controller:
@RestController
@RequestMapping(value = "/api/event")
@RequiredArgsConstructor
public class EventHandler {
private final EventRepository repo;
@PostMapping
public Mono<String> create(Event event) {
return Mono.just(event.withNewId().getId())
.doOnNext(id ->
// do query based on some logic depending on event data
Mono.just(someQuery)
.flatMap(query ->
repo.find(query)
.map(e -> event.setData(event.getData() + e.getData())))
.switchIfEmpty(Mono.just(event))
.flatMap(e -> repo.save(e))
.subscribeOn(Schedulers.single())
.subscribe());
}
}
It does not work, but with subscribeOn
I try to guarantee linearizability.它不起作用,但使用
subscribeOn
我试图保证线性化。 Meaning that concurrent requests A and B will always write their payload to the DB in the order in which they are received by the server.这意味着并发请求 A 和 B 将始终按照服务器接收它们的顺序将它们的有效负载写入数据库。 Therefore if another concurrent request C is a compound of first read than write, it will read changes from the DB that reflect those of request B, not A, and write its own changes based of B.
因此,如果另一个并发请求 C 是先读后写的复合请求,它将从数据库中读取反映请求 B 而不是 A 的更改,并根据 B 写入自己的更改。
Is there a way in Reactor Netty to schedule executors with an unbound FIFO queue, so that I can process the requests asynchronously but in order? Reactor Netty 中是否有一种方法可以使用未绑定的 FIFO 队列来安排执行程序,以便我可以异步但按顺序处理请求?
I don't think that this is specific to Netty or Reactor in particular, but to a more broad topic - how to handle out-of-order message delivery and more-than-once message delivery.我认为这不是特定于 Netty 或 Reactor,而是更广泛的主题——如何处理乱序消息传递和多次消息传递。 A few questions:
几个问题:
I'd try to redesign the operation in such a way that there's a single request executing the operations on the backend in the required order and using concurrency here if necessary to speed-up the process.我会尝试以这样一种方式重新设计操作,即有一个请求按要求的顺序在后端执行操作,并在必要时在此处使用并发来加快流程。
If it's not possible, for example, you don't control the client, or more generally the order in which the events (requests) arrive, you have to implement ordering on application-level logic using per-message semantics to do the ordering.如果不可能,例如,您不控制客户端,或者更一般地控制事件(请求)到达的顺序,您必须使用每消息语义在应用程序级逻辑上实现排序来进行排序。 You can, for example store or buffer the messages, waiting for all to arrive, and when they do, only then trigger the business logic using the data from the messages in the correct order.
例如,您可以存储或缓冲消息,等待所有消息到达,当它们到达时,才以正确的顺序使用消息中的数据触发业务逻辑。 This requires some kind of a key (identity) which can attribute messages to the same entity, and a sorting-key, that you know how to sort the messages in the correct order.
这需要某种可以将消息归因于同一实体的键(身份)和一个排序键,您知道如何以正确的顺序对消息进行排序。
EDIT: After getting the answers, you can definitely implement it "the Reactor way".编辑:得到答案后,您绝对可以“反应堆方式”实施它。
Sinks.Many<Event> sink = Sinks.many() // you creat a 'sink' where the events will go
.multicast() // broads all messages to all subscribes of the stream
.directBestEffort(); // additional semantics - publishing will fail if no subscribers - doesn't really matter here
Flux<Event> eventFlux = sink.asFlux(); // the 'view' of the sink as a flux you can subscribe to
public void run() {
subscribeAndProcess();
sink.tryEmitNext(new Event("A", "A", "A"));
sink.tryEmitNext(new Event("A", "C", "C"));
sink.tryEmitNext(new Event("A", "B", "B"));
sink.tryEmitNext(new Event("B", "A", "A"));
sink.tryEmitNext(new Event("B", "C", "C"));
sink.tryEmitNext(new Event("B", "B", "B"));
}
void subscribeAndProcess() {
eventFlux.groupBy(Event::key)
.flatMap(
groupedEvents -> groupedEvents.distinct(Event::type) // distinct to avoid duplicates
.buffer(3) // there are three event types, so we buffer and wait for all to arrive
.flatMap(events -> // once all the events are there we can do the processing the way we need
Mono.just(events.stream()
.sorted(Comparator.comparing(Event::type))
.map(e -> e.key + e.value)
.reduce(String::concat)
.orElse(""))
)
)
.subscribe(System.out::println);
}
// prints values concatenated in order per key:
// - AAABAC
// - BABBBC
See Gist: https://gist.github.com/tarczynskitomek/d9442ea679e3eed64e5a8470217ad96a见要点: https://gist.github.com/tarczynskitomek/d9442ea679e3eed64e5a8470217ad96a
There are a few caveats:有几点需要注意:
Having all this in mind, I would go with a persistent storage - say saving the incoming events in the database, and doing the processing in background - for this you don't need to use Reactor.考虑到所有这些,我会使用持久存储 go - 比如说将传入的事件保存在数据库中,并在后台进行处理 - 为此你不需要使用 Reactor。 Most of the time a simple Servlets based Spring app will be far easier to maintain and develop, especially if you have no previous experience with Functional Reactive Programming.
大多数时候,基于 Spring 的简单 Servlets 应用程序将更容易维护和开发,尤其是如果您之前没有使用函数式响应式编程的经验。
Looking at the provided code I would not try to handle it on Reactor Netty level.查看提供的代码,我不会尝试在 Reactor Netty 级别上处理它。
At first, several comments regarding controller implementation because it has multiple issues that violate reactive principles.首先,关于 controller 实现的几个评论,因为它有多个问题违反了反应原则。 I would recommend to spend some time learning reactive API but here are some hints
我建议花一些时间学习反应式 API 但这里有一些提示
In reactive nothing happens until you subscribe.在你订阅之前,反应式不会发生任何事情。 At the same time calling
subscribe
explicitly is an anti-pattern and should be avoided until you are creating framework similar to WebFlux.同时显式调用
subscribe
是一种反模式,在创建类似于 WebFlux 的框架之前应避免使用。
parallel
scheduler should be used to run non-blocking logic until you have some blocking code. parallel
调度程序应该用于运行非阻塞逻辑,直到你有一些阻塞代码。
doOn...
are so-called side-effect operators and should not be used for constructing reactive flows. doOn...
是所谓的副作用运算符,不应用于构建反应流。
@PostMapping
public Mono<String> create(Event event) {
// do query based on some logic depending on event data
return repo.find(query)
.map(e -> event.setData(event.getData() + e.getData()))
.switchIfEmpty(Mono.just(event))
.flatMap(e -> repo.save(e));
}
Now, processing requests in the predefined sequence could be tricky because of.network failures, possible retries, etc. What if you never get Request B or Request C?现在,由于网络故障、可能的重试等原因,按预定义顺序处理请求可能会很棘手。如果您永远不会收到请求 B 或请求 C 怎么办? Should you still persist Request A?
你还应该坚持请求A吗?
As @ttarczynski mentioned in his comment the best option is to redesign API and send single request.正如@ttarczynski 在他的评论中提到的,最好的选择是重新设计 API 并发送单个请求。
In case it's not an option you would need to introduce some state to "postpone" request processing and then, depending on consistency semantic, process them as a "batch" when the last request is received or just defer Request C until you get Request A & B.如果这不是一个选项,您需要引入一些 state 来“推迟”请求处理,然后根据一致性语义,在收到最后一个请求时将它们作为“批处理”处理,或者只是推迟请求 C 直到获得请求 A & B。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.