Kafka sink mongoDB

Question

From what I've seen of examples for performing a SINK for a Mongo database, I need to define a single SINK config for each collection that I want to save to the Mongo. The problem is that I have 61 topics in kafka and I would like to create 61 collections in MongoDB, but it bothers me to have to create 61 settings for SINK. Is there any alternative way? In databases like MySQL when I do SINK, all topics automatically create tables (61 tables) and I use a single SINK configuration. I would like to have something similar for Mongo. Can you help me?

Answer 1

In the Sink connector's property file you can use topics.regex option to whitelist multiple topics.

you can pass your regex like this

topics.regex = ^[A-Za-z._-]+$

Or If you have topics like

ABC.Topics1
ABC.Topics2
ABC.Topics3

you can pass regex like this

topics.regex = ^(ABC.)([a-z._-]+)+$