[英]Using Kafka Streams just as a state store in a Kafka Consumer App
I am currently working on a Spring boot application using Spring Kafka Consumer API. 我目前正在使用Spring Kafka Consumer API开发Spring Boot应用程序。
Each message I get on to a topic needs to be converted into a new object type with additional properties coming in from other topics. 我到达某个主题的每条消息都需要转换为新的对象类型,并具有来自其他主题的其他属性。 Currently, these other topics are not yet developed and we are using a mocked version of in-memory data for processing the requests.
目前,这些其他主题尚未开发,我们正在使用内存数据的模拟版本来处理请求。
For example, a new "shopping order" message arrives, but I am using mocked "Customer" object and mocked "item" object in order to process the order. 例如,一条新的“购物订单”消息到达,但是我正在使用模拟的“客户”对象和模拟的“项目”对象以处理订单。 The plan is to move to use real Customer topic and real Item topic.
计划是使用真实的客户主题和真实的项目主题。
Also, currently, the application is only Spring Kafka listeners for getting new order. 另外,当前,该应用程序仅是用于获得新订单的Spring Kafka侦听器。 The listeners invoke a spring bean method which processes the order and creates a new object to be written to another output topic named customer-order by using the same mocks I mentioned above.
侦听器调用一个spring bean方法,该方法处理订单并创建一个新对象,该对象使用与我上面提到的相同的模拟方法写入另一个名为customer-order的输出主题。
We are currently thinking about evolving the architecture for this application. 我们目前正在考虑改进此应用程序的体系结构。 I have been reading up on Kafka streams.
我一直在阅读卡夫卡流。 The documentation I have read online for streams only take simple examples such as word count, join etc. With my limited knowledge of streams, I don't envision using functionality such as calculating total etc.
我在网上阅读的关于流的文档仅采用简单的示例,例如字数统计,联接等。由于对流的了解有限,因此我不打算使用诸如计算总数之类的功能。
I have thought of some options for the architecture... 我已经想到了该体系结构的一些选择...
What do you suggest? 你有什么建议? 1,2, or 3?
1,2还是3? Is that a good idea to use Streams for this kind of solution?
将Streams用于这种解决方案是一个好主意吗? Is there any benefit in moving this implementation to use Kafka streaming?
将这种实现方式转换为使用Kafka流媒体有什么好处? Or am I better of staying with 2.?
还是我最好还是和2呆在一起?
Number 1 sounds strange to me. 1对我来说听起来很奇怪。 You can keep a KafkaStreams application exposing state stores via Interactive Queries, but that would look more like a flavour of 2. You'll have to take into account also how you deploy your instances and ensure co-partitioning between the Spring part and the KafkaStreams part.
您可以保留一个KafkaStreams应用程序通过Interactive Queries公开状态存储,但是看起来更像是2。您还必须考虑如何部署实例并确保Spring部分和KafkaStreams之间的共分区。部分。
I don't see any problem in doing it in Kafka Streams fully, unless you have some very complex logic you cannot implement with current API, which I'd be surprised to learn you couldn't. 我完全不会在Kafka Streams中完成任何操作,除非您有一些非常复杂的逻辑,无法使用当前的API来实现,但如果您无法做到这一点,我会感到很惊讶。 Actually what you described sounds like a usual application for it (with the caveat of not knowing other requirements like time, expected volumes, etc).
实际上,您所描述的内容听起来像是它的常规应用程序(警告是不了解其他需求,例如时间,预期数量等)。
Benefits: 优点:
Cons: 缺点:
This is not a comprehensive list, but it's the most important points from the top of my head. 这不是一个完整的列表,但这是我脑海中最重要的一点。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.