將數據從 kafka 復制到 mysql，無法使用 DOcker 和 Debezium 連接到 JDBCSinkConnector

Question

嗨，我正在使用 debezium 捕獲 Mongo 中的更改並將它們推送到 mysql 我正在使用以下鏈接https://github.com/debezium/debezium-examples/tree/master/unwrap-mongodb-smt我正在替換結束postgres db 到 mysql 數據庫，但我無法這樣做。

這是我修改后的 jdbc-sink.json，這里我使用 mysql url 進行連接。

{
    "name" : "jdbc-sink",
    "config" : {
        "connector.class":"io.confluent.connect.jdbc.JdbcSinkConnector",
        "tasks.max" : "1",
        "topics" : "customers",
        "connection.url" : "jdbc:mysql://localhost:3306/inventorydb?user=user&password=password",
        "auto.create" : "true",
        "auto.evolve" : "true",
        "insert.mode" : "upsert",
        "delete.enabled": "true",
        "pk.fields" : "id",
        "pk.mode": "record_key",
        "transforms": "mongoflatten",
        "transforms.mongoflatten.type" : "io.debezium.connector.mongodb.transforms.ExtractNewDocumentState",
        "transforms.mongoflatten.drop.tombstones": "false"
    }
}

但是我在運行時收到以下錯誤

curl -i -X POST -H "Accept:application/json" -H  "Content-Type:application/json" http://localhost:8083/connectors/ -d @jdbc-sink.json

HTTP/1.1 500 內部服務器錯誤日期：2019 年 11 月 6 日星期三 08:13:39 GMT 內容類型：application/json 內容長度：3404 服務器：Jetty(9.4.18.v20190429)

{" error_code":500,"message":"Failed to find any class that implements Connector and which name matches io.confluent.connect.jdbc.JdbcSinkConnector , available connectors are: PluginDesc{klass=class io.debezium.connector.mongodb .MongoDbConnector, name='io.debezium.connector.mongodb.MongoDbConnector', version='1.0.0-SNAPSHOT', encodedVersion=1.0.0-SNAPSHOT, type=source, typeName='source', location='file: /kafka/connect/debezium-connector-mongodb/'}, PluginDesc{klass=class io.debezium.connector.mysql.MySqlConnector, name='io.debezium.connector.mysql.MySqlConnector', version='1.0.0- SNAPSHOT'，encodedVersion=1.0.0-SNAPSHOT，type=source，typeName='source'，location='file:/kafka/connect/debezium-connector-mysql/'}，PluginDesc{klas s=class io.debezium.connector.oracle.OracleConnector, name='io.debezium.connector.oracle.OracleConnector', version='1.0.0-SNAPSHOT', encodedVersion=1.0.0-SNAPSHOT, type=source, typeName ='source', location='file:/kafka/connect/debezium-connector-oracle/'}, PluginDesc{klass=class io.debezium.connector.postgresql.PostgresConnector, name='io.debezium.connector.postgresql. PostgresConnector'，版本='1.0.0-SNAPSHOT'，encodedVersion=1.0.0-SNAPSHOT，type=source，typeName='source'，location='file:/kafka/connect/debezium-connector-postgres/'}， PluginDesc{klass=class io.debezium.connector.sqlserver.SqlServerConnector, name='io.debezium.connector.sqlserver.SqlServerConnector', version='1.0.0-SNAPSHOT', encodedVersion=1.0.0-SNAPSHOT, type=source , typeName='source', location='file:/kafka/connect/debezi um-connector-sqlserver/'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.2.3.0SinkConnector',', =' encodedVersion=2.3.0，type=sink，typeName='sink'，location='classpath'}，PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector，name='org.ZB6EFD606D118D0F62066E31419FFf04CCZka. .file.FileStreamSourceConnector'，版本='2.3.0'，encodedVersion=2.3.0，type=source，typeName='source'，location='classpath'}，PluginDesc{klass=class org.apache.kafka.connect。 tools.MockConnector，name='org.apache.kafka.connect.tools.MockConnector'，version='2.3.0'，encodedVersion=2.3.0，type=connector，typeName='connector'，location='classpath'} , PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.ZB6E FD606D118D0F62066E31419FF04CCZ.kafka.connect.tools.MockSinkConnector'，版本='2.3.0'，encodedVersion=2.3.0，type=sink，typeName='sink'，location='classpath'}，PluginDesc{klass=class org.ZB6EFD606D118D0F62066E314Z19FF0 .kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='2.3.0', encodedVersion=2.3.0, type=source, typeName='source', location ='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='2.3.0', encodedVersion2. .0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect. VerifiableSinkConnector'，版本='2.3.0'，編碼版本=2.3.0，類型=來源，類型名稱='來源' , location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='2'. =2.3.0, type=source, typeName='source', location='classpath'}"}

我知道有些它無法找到 io.confluent.connect.jdbc.JdbcSinkConnector 但我應該如何/我應該在哪里保存這樣的 Z68995FCBF432492D15484D04A9D2AC0

謝謝

Answer 1

您還沒有在 Kafka Connect 中提供 sink 連接器，請參閱命令docker-compose up --build -d用於開始構建新的 Connect 映像，其中 JDBC sink 連接器在Z5E056C500A1C4B6A7110B50A7/BADEZium7/BADEBZium7中烘焙。 示例/blob/master/unwrap-mongodb-smt/debezium-jdbc/Dockerfile#L10

Answer 2

使用以下命令下載 JAR 並將其保存在插件目錄中：

curl -sO https://packages.confluent.io/maven/io/confluent/kafka-connect-jdbc/10.0.0/kafka-connect-jdbc-10.0.0.jar