洋葱架构中的 Kafka 消费者

Question

I am working with a project following DDD and has consumers to a Kafka Queue . My question is straight forward, where do the consumers reside in the Onion Architecture and Hexagonal Architecture ? Are they event handlers or should they be a part of the infrastructure?

I am using Kafka Consumers to listen to change events of other aggregate roots and want to store the data in my current aggregate . Basically replicating the data from one microservice to another.

Answer 1

The purpose of a (Kafka) consumer is very similar to the purpose of eg a RESTful endpoint - they both consume messages/requests from the outside world in order to convert them to something understandable by the Application . So, from Hexagonal Architecture point of view, a (Kafka) consumer is just an adapter . Check also the drawing at page 148 from Microservices Patterns by Chris Richardson where a queue consumer (aka handler ) is placed as an adapter in the context of the Hexagonal Architecture .

When thinking about Clean Architecture focus a bit on the mention about the flow control:

It begins in the controller , moves through the use case, and then winds up executing in the presenter.

A (Kafka) consumer is the entry point for the outside messages, the flow control begins with it because there's nothing else to begin with and it basically behaves like a Controller ; thus the (Kafka) consumer is part of Interface Adapters layer, Controllers section (see the image ).

When considering Onion Architecture , I would compare the (Kafka) consumer to a Controller : both do the same thing, mainly receiving a request, despite the fact that the request's transport mechanism might differ. I consider that both should stay in the same place which is User Interface segment of the edge layer (notice the SpeakerController ).

About frameworks

Depending on the used framework, a message consumer might have nothing to directly do with external requests, instead it might directly receive some identifiers or even some domain object(s). Eg when using @KafkaListener from Spring for Apache Kafka the consumer could actually be an Use Case (Clean Architecture) or an Application Service (Onion Architecture); it could also be a domain service (Onion Architecture) and even an entity (very unlikely) when using some (Spring framework) tricks.

Conclusion

I think that the key to answer the post's question is to ask it this way: “is the consumer implementation depending on the infrastructure implementation?”. If the answer is “yes” than one would at least know that the consumer implementation can't stay in the code/domain/model part of the application; if the answer is “no” (see “About frameworks” above) than the consumer implementation might be part of code/domain/model .

Answer 2

The way I see it is:

1. Your aggregates are the Core

2. The message handlers are Use Cases, which use dependencies (like repository interfaces) and the core to execute the business use case.

3. There is infrastructure code that peeks messages from the queue and triggers use cases

With this approach, you can unit test your use cases without worrying about infrastructure and you can replace the whole messaging queue technology. The same approach works for handling API requests. In practice, the only difference is that with the API you can return a response synchronously, and with messaging you can't.

As a practical note, in .NET I use a library called Mediatr to implement and trigger the use cases. In java, I found PipelinR which looks similar at first sight. This type of approach allows you to implement all use cases the same way for all your synchronous and asynchronous usages.