将 Kafka 连接嵌入到 Ksqldb-server 时挂载（卷）不起作用

Question

The "volumes" mapping does not seem to work when trying to embed Kafka connector in Ksqldb Server

Below is my docker file

ksqldb-server:
        image: confluentinc/ksqldb-server:0.18.0
        hostname: ksqldb-server
        container_name: ksqldb-server
        depends_on:
          - broker
          - schema-registry
        ports:
          - "8088:8088"
        volumes:
          - "/usr/local/share/kafka/connectors/debezium-debezium-connector-mysql/lib:/usr/share/kafka/plugins/"
        environment:
          KSQL_LISTENERS: "http://0.0.0.0:8088"
          KSQL_BOOTSTRAP_SERVERS: "broker:9092"
          KSQL_KSQL_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
          KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
          KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
          # Configuration to embed Kafka Connect support.
          KSQL_CONNECT_GROUP_ID: "ksql-connect-cluster"
          KSQL_CONNECT_BOOTSTRAP_SERVERS: "broker:9092"
          KSQL_CONNECT_KEY_CONVERTER: "org.apache.kafka.connect.storage.StringConverter"
          KSQL_CONNECT_VALUE_CONVERTER: "io.confluent.connect.avro.AvroConverter"
          KSQL_CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
          KSQL_CONNECT_CONFIG_STORAGE_TOPIC: "_ksql-connect-configs"
          KSQL_CONNECT_OFFSET_STORAGE_TOPIC: "_ksql-connect-offsets"
          KSQL_CONNECT_STATUS_STORAGE_TOPIC: "_ksql-connect-statuses"
          KSQL_CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
          KSQL_CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
          KSQL_CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1
          KSQL_CONNECT_PLUGIN_PATH: "/usr/share/kafka/plugins"

I see the below error when I am trying to create a source using the connector

{ "error_code" : 500, "message" : "Failed to find any class that implements Connector and which name matches io.debezium.connector.mysql.MySqlConnector, available connectors are: ..........}" }

I do have the required jar files installed locally in the folder "/usr/local/share/kafka/connectors/debezium-debezium-connector-mysql/lib" as mapped in the volumes in the docker file above.

What is it that I am missing?

Much appreciated

Answer 1

After downloading a connector zip file extract their files into the plugin path in your worker configuration (eg connect-distributed.properties) using the plugin.path configuration property. As an example, let's assume you have downloaded the Debezium MySQL connector archive and extracted its contents to /kafka/connect/debezium-connector-mysql. Then you'd specify the following in the worker config:

plugin.path=/kafka/connect

So in your case you mounting to an inner path, the connector plugin needs to be inside another folder and not directly the plugin path mount