简体   繁体   中英

Confluent Kafka Connect Elasticsearch connector installation

I'm trying to install Elasticsearch connector to Confluent Kafka Connect. I'm following below instruction: https://docs.confluent.io/kafka-connect-elasticsearch/current/index.html#install-the-connector-using-c-hub
after executing:

confluent-hub install confluentinc/kafka-connect-elasticsearch:latest

everything seems fine. See below result:

[ec2-user@ip-172-31-16-76 confluent-6.1.0]$ confluent-hub install confluentinc/kafka-connect-elasticsearch:latest
The component can be installed in any of the following Confluent Platform installations:
  1. /home/ec2-user/confluent-6.1.0 (based on $CONFLUENT_HOME)
  2. /home/ec2-user/confluent-6.1.0 (found in the current directory)
  3. /home/ec2-user/confluent-6.1.0 (where this tool is installed)
Choose one of these to continue the installation (1-3): 2
Do you want to install this into /home/ec2-user/confluent-6.1.0/share/confluent-hub-components? (yN) y


Component's license:
Confluent Community License
http://www.confluent.io/confluent-community-license
I agree to the software license agreement (yN) y

Downloading component Kafka Connect Elasticsearch 11.0.3, provided by Confluent, Inc. from Confluent Hub and installing into /home/ec2-user/confluent-6.1.0/share/confluent-hub-components
Do you want to uninstall existing version 11.0.3? (yN) y

Detected Worker's configs:
  1. Standard: /home/ec2-user/confluent-6.1.0/etc/kafka/connect-distributed.properties
  2. Standard: /home/ec2-user/confluent-6.1.0/etc/kafka/connect-standalone.properties
  3. Standard: /home/ec2-user/confluent-6.1.0/etc/schema-registry/connect-avro-distributed.properties
  4. Standard: /home/ec2-user/confluent-6.1.0/etc/schema-registry/connect-avro-standalone.properties
  5. Based on CONFLUENT_CURRENT: /tmp/confluent.424339/connect/connect.properties
  6. Used by Connect process with PID : /tmp/confluent.424339/connect/connect.properties
Do you want to update all detected configs? (yN) y

Adding installation directory to plugin path in the following files:
  /home/ec2-user/confluent-6.1.0/etc/kafka/connect-distributed.properties
  /home/ec2-user/confluent-6.1.0/etc/kafka/connect-standalone.properties
  /home/ec2-user/confluent-6.1.0/etc/schema-registry/connect-avro-distributed.properties
  /home/ec2-user/confluent-6.1.0/etc/schema-registry/connect-avro-standalone.properties
  /tmp/confluent.424339/connect/connect.properties
  /tmp/confluent.424339/connect/connect.properties

Completed

However, when I'm trying to list all avilable connectors I'm getting below list:

[ec2-user@ip-172-31-16-76 confluent-6.1.0]$ confluent local services connect connector list
The local commands are intended for a single-node development environment only,
NOT for production usage. https://docs.confluent.io/current/cli/index.html

Bundled Connectors:
  file-sink
  file-source
  replicator

As per instruction in link above I would expect to see elasticsearch-sink . Unofrtunetly, no such entry avilable.
It seems I'm missing something simple but I don't see any explenation in instruction. Any help would be appreciated.

EDIT 1
Below you can see result of curl -s localhost:8083/connector-plugins

[
    {"class":"io.confluent.connect.elasticsearch.ElasticsearchSinkConnector","type":"sink","version":"11.0.3"},
    {"class":"io.confluent.connect.replicator.ReplicatorSourceConnector","type":"source","version":"6.1.0"},
    {"class":"io.confluent.kafka.connect.datagen.DatagenConnector","type":"source","version":"null"},
    {"class":"org.apache.kafka.connect.file.FileStreamSinkConnector","type":"sink","version":"6.1.0-ce"},
    {"class":"org.apache.kafka.connect.file.FileStreamSourceConnector","type":"source","version":"6.1.0-ce"},
    {"class":"org.apache.kafka.connect.mirror.MirrorCheckpointConnector","type":"source","version":"1"},
    {"class":"org.apache.kafka.connect.mirror.MirrorHeartbeatConnector","type":"source","version":"1"},
    {"class":"org.apache.kafka.connect.mirror.MirrorSourceConnector","type":"source","version":"1"}
]

curl -s localhost:8083/connector-plugins gives the definitive response from the worker what plugins are installed.

Per the output in your question, the Elasticsearch sink connector is now installed in your connector. I don't know why the Confluent CLI would not show this.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM