Use Confluent Hub without Confluent Platform installation

Question

I am using these installation instructions for getting confluent hub client https://docs.confluent.io/current/connect/managing/confluent-hub/client.html

But, when I get to the line to install the kafka-connect-elasticsearch connector

confluent-hub install confluentinc/kafka-connect-elasticsearch:latest

I continue to get this error message:

Unable to detect Confluent Platform installation. Specify --component-dir and --worker-configs explicitly.

Error: Invalid options or arguments

I have ElasticSearch and Kafka all installed via Homebrew on my Mac

Answer 1

Specify --component-dir and --worker-configs explicitly

You'll have to pass both options to include the Homebrew location of Kafka Connect's plugin.path setting (where Confluent Hub would install to) and your connect-distributed.properties (so that the plugin path can be setup if it isn't already)

eg

confluent-hub install <name> \
   --component-dir /opt/connectors \
   --worker-configs /etc/kafka/connect-distributed.properties

This command should download and extract the connector zip into /opt/connectors and modify the connect-distributed.properties file to include that path in its plugin.path property

Here is an example using a Kafka Connect docker image that I've built

Answer 2

MacOS

In my case I have installed Kafka and Confluent Hub with brew and the installation path for Kafka is /usr/local/Cellar/kafka . And created a directory to install the connector.

Move to /usr/local/Caskroom/confluent-hub-client/6.2.1/bin , it is the directory confluent-hub is located.

Then I've executed confluent-hub install confluentinc/kafka-connect-azure-blob-storage:1.6.2 --component-dir /Users/Rustam_Zokirov/BigData/connectors --worker-configs /usr/local/Cellar/kafka/3.0.0/libexec/config/connect-distributed.properties

Answer 3

我有同样的使用本地融合社区版本，您应该在包含融合平台的目录中启动安装（与 bin 等相同，共享...）手动指定配置文件可能会出错。