[英]Setting Kafka Consumer Level Configuration in Docker Compose File
I have a situation where I have two docker compose files: one contains, among other things, a Kafka image;我有两个 docker 撰写文件的情况:其中一个包含 Kafka 图像; the other contains some other process which consumes from Kafka.
另一个包含一些从 Kafka 消耗的其他进程。
kakfa.yaml
: kakfa.yaml
:
version: "3.4"
services:
zookeeper:
image: "wurstmeister/zookeeper:latest"
hostname: zookeeper
ports:
- "2181:2181"
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
restart: on-failure
kafka:
image: "wurstmeister/kafka:latest"
hostname: kafka
ports:
- "9092:9092"
- "9093:9093"
- "19092:19092"
environment:
KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
KAFKA_ADVERTISED_PORT: "9092"
KAFKA_LISTENERS: "INTERNAL://kafka:9092,EXTERNAL://0.0.0.0:9093"
KAFKA_ADVERTISED_LISTENERS: "INTERNAL://kafka:9092,EXTERNAL://localhost:9093"
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT"
KAFKA_INTER_BROKER_LISTENER_NAME: "INTERNAL"
depends_on:
- zookeeper
restart: on-failure
processor.yaml
version: "3.4"
services:
processor:
build: .
environment:
- LOGURU_LEVEL=DEBUG
- MQ_BOOTSTRAP_SERVER=kafka
- MQ_BOOTSTRAP_PORT=9092
ports:
- "8801:80"
restart: on-failure
volumes:
- ./service:/app/service
entrypoint: [ '/bin/sh', '-c' ]
command: |
"
kafka-console-consumer --bootstrap-server localhost:9092 --topic example_topic
"
My command
attempt at the bottom of processor.yaml
gives the following error in docker: /bin/sh: 2: kafka-console-consumer: not found
.我在
processor.yaml
底部的command
尝试在 docker: /bin/sh: 2: kafka-console-consumer: not found
中给出以下错误。
Is there a way I can set Consumer configs within processor.yaml
?有没有办法可以在
processor.yaml
中设置消费者配置?
Edit: my dockerfile for both yaml files编辑:我的 dockerfile 两个 yaml 文件
ENV APP_PATH=/app
ENV PYTHONPATH="${PYTHONPATH}:${APP_PATH}"
COPY requirements.txt "${APP_PATH}/requirements.txt"
RUN pip install --no-cache-dir -r "${APP_PATH}/requirements.txt"
COPY . $APP_PATH
WORKDIR $APP_PATH
Your error is not related to "consumer configs".您的错误与“消费者配置”无关。 Your Dockerfile appears to be a Python container.
您的 Dockerfile 似乎是 Python 容器。 You have not installed the Kafka CLI tools there such that
kafka-console-consumer.sh
command would be available.您尚未在那里安装 Kafka CLI 工具,因此
kafka-console-consumer.sh
命令可用。
To have the Kafka CLI tools, you'd also need to install Java in that container, which would cause your image to be larger than necessary.要拥有 Kafka CLI 工具,您还需要在该容器中安装 Java,这会导致您的图像超出必要的大小。 Instead, you should write a consumer in a Python client, then run your own
python consumer.py
as your container command.相反,您应该在 Python 客户端中编写消费者,然后运行您自己的
python consumer.py
作为您的容器命令。
Also, localhost:9092
is not referring to the Kafka container.此外,
localhost:9092
并不是指 Kafka 容器。 Related - Connect to Kafka running in Docker相关 - 连接到在 Docker 中运行的 Kafka
Otherwise, you don't need a different container to run kafka-console-consumer
;否则,您不需要其他容器来运行
kafka-console-consumer
; you can docker-compose exec kafka bash -c "kafka-console-consumer --bootstrap-server localhost:9093... "
你可以
docker-compose exec kafka bash -c "kafka-console-consumer --bootstrap-server localhost:9093... "
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.