我可以在单个 Kafka-Connector S3 接收器连接器中映射具有多个主题的多个存储桶吗？

Question

Google didn't help me, so I want to ask you. I have a lot of kafka topics, and I want to store the messages of a particular topic in a particular S3 bucket.

Do I need to create an S3 sink connector for each bucket or can I configure all the stuff in a single S3 json file (map a bucket with a topic)?

Thanks for your help! :)

Answer 1

The S3 bucket name is specified per-connector , so you will need to create one connector per bucket.

Note that you can specify multiple topics per connector ( topics / topics.regex ), so if you wanted multiple topics going to one bucket (but different paths), you could do this with a single connector. But for multiple buckets, you'll need multiple connectors.