I'm running kafka locally via:
docker-compose.yml
zookeeper:
image: 'bitnami/zookeeper:latest'
ports:
- 2181:2181
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
kafka:
image: 'bitnami/kafka:latest'
ports:
- 9092:9092
environment:
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_ADVERTISED_PORT=9092
- KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092
My Spring Boot application is run with:
application.yml:
spring:
application:
name: testkafka
kafka:
bootstrap-servers: localhost:9092
server:
port: 8080
When I run it and it tries to send to a topic on kafka, I get:
org.springframework.kafka.KafkaException: Reply timed out
at org.springframework.kafka.requestreply.ReplyingKafkaTemplate.lambda$sendAndReceive$0(ReplyingKafkaTemplate.java:196) ~[spring-kafka-2.1.10.RELEASE.jar:2.1.10.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.0.10.RELEASE.jar:5.0.10.RELEASE]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) ~[na:na]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java) ~[na:na]
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
If I run the spring boot from inside a docker container (using one docker compose file), then it does work:
Running Both in the Same Compose:
version: "3.0"
services:
service1:
build: ./Service
ports:
- 8080:8080
- 5005:5005
links:
- zookeeper
- kafka
zookeeper:
image: 'bitnami/zookeeper:latest'
ports:
- 2181:2181
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
kafka:
image: 'bitnami/kafka:latest'
ports:
- 9092:9092
environment:
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_ADVERTISED_PORT=9092
- KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092
How can I get the kafka container to allow connections from outside itself/docker?
EDIT: Tried these changes:
kafka:
image: 'bitnami/kafka:latest'
ports:
- 9092:9092
environment:
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_ADVERTISED_PORT=9092
- KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092
and:
spring:
application:
name: testkafka
kafka:
bootstrap-servers: kafka:9092
server:
port: 8080
This still times out
If I run the spring boot from inside a docker container (using one docker compose file), then it does work
It shouldn't work, actually. Kafka isn't running as part of the application, so this section is not pointing at the Kafka container.
kafka:
bootstrap-servers: localhost:9092
It would need to be kafka:9092
within the Docker network.
and ...
bootstrap-servers: kafka:9092
... still times out
That's because you still have KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092
Both inside and outside of Docker network, by specifying KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092
, you're saying that your clients receive the Kafka bootstrap connection as localhost:9092
, which would work outside of the Docker network only because you have made both a port forwarding and your container is running on localhost, however, inside the Docker network, as mentioned, localhost would mean that application container, not the broker.
The solution would be to create two port mappings via that property, as discussed in length by this blog post
Plus, Confluent provides a fully configured Docker Compose with appropriate mappings that'll work inside and out of Docker
ports:
- "9092:9092"
- "29092:29092"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
ALLOW_PLAINTEXT_LISTENER: "yes"
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_LISTENERS: PLAINTEXT://:9092,PLAINTEXT_HOST://0.0.0.0:29092
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
For applications within Docker network, use kafka:9092
, for applications outside, use localhost:29092
EDIT The bitnami config variables have changed. Refer this section of the README
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.