从 MongoDB 到主题的字段

Question

I want to run a Kafka Connect as a Source with the required fields from Mongo, and not pull together the entire collection. Is this possible? For Example: I have collection

 { contact_name: "Ace Farad", contact_uid: "HdeofVad" }

but to topic I need to send only contact_name. Thanks!

Answer 1

Use a transform to drop a field

"transforms": "ReplaceField",
"transforms.ReplaceField.type": "org.apache.kafka.connect.transforms.ReplaceField$Value",
"transforms.ReplaceField.blacklist": "contact_uid"

https://docs.confluent.io/platform/current/connect/transforms/replacefield.html

Note: this may only work for top level fields, not nested collections

The alternative is to read all the data into a "raw" topic, then manipulate it how you want inside of Kafka