简体繁体 English

spring boot中如何从Mongodb读取采集数据并定期发布到kafka topic

[英]How to read collection data from Mongodb and publish into kafka topic periodically in spring boot

原文 2020-10-02 15:04:28 5 1 mongodb/ spring-boot/ spring-batch/ spring-kafka

I need to read my mongo DB table data periodically and publish it into a Kafka topic using spring boot.我需要定期读取我的 mongo DB 表数据并使用 Spring Boot 将其发布到 Kafka 主题中。 I have created a collection in Mongo DB and inserted a few records in Mongo DB.我在 Mongo DB 中创建了一个集合，并在 Mongo DB 中插入了一些记录。 Further, I want to read the data from Mongo DB periodically and need to publish those table data in Kafka's topic using spring boot.此外，我想定期从 Mongo DB 读取数据，并需要使用 Spring Boot 在 Kafka 的主题中发布这些表数据。 I'm very new to spring batch scheduler.我对 spring 批处理调度程序很陌生。 Can you please suggest me an idea to achieve this?你能建议我一个想法来实现这一目标吗？

Thanks in advance.提前致谢。

1 个解决方案

What you are talking about is more relevant to Spring Integration: https://spring.io/projects/spring-integration#overview您所说的与 Spring Integration 更相关： https : //spring.io/projects/spring-integration#overview

So, you configure a MongoDbMessageSource with a Poller to read collection periodically.因此，您使用Poller配置MongoDbMessageSource以定期读取集合。 And then you have service-activator based on the KafkaProducerMessageHandler to damp data into a Kafka topic.然后你有基于KafkaProducerMessageHandler服务激活KafkaProducerMessageHandler来将数据放入 Kafka 主题中。

See more in docs:在文档中查看更多信息：

https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/mongodb.html#mongodb https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/mongodb.html#mongodb

https://docs.spring.io/spring-integration/docs/5.4.0-M3/reference/html/kafka.html#kafka https://docs.spring.io/spring-integration/docs/5.4.0-M3/reference/html/kafka.html#kafka

Not sure though how to do that with Spring Batch...不确定如何使用 Spring Batch 做到这一点......