将 Kafka 连接 MongoDB 连接器注册为 SCDF 中的应用程序

Question

I'll get straight to the point :

Is it possible to register the MongoDB Kafka sink and source connectors as applications in Spring Cloud Data Flow ? or other types of Kafka connectors in the matter of fact ?

MongoDB Kafka Source connector requires MongoDB to be configured as a replicaset cluster in order to read the change streams in the opLog (a standalone MongoDB instance cannot produce a change stream). Does the SCDF MongoDB source starter use MongoDb cluster's change streams to detect change events ? or does it read changes directly from the MongoDB database ?

Thanks

Answer 1

We have looked at integrating Spring Cloud Stream with Kafka connectors. There is no easy way to do it without custom code. We have a change data capture source that works with MongoDB as you describe: https://github.com/spring-cloud/stream-applications/blob/main/functions/supplier/cdc-debezium-supplier/README.adoc