Spring Application cannot connect to Kafka when it's run inside of the same cluster, but works when it runs from outside cluster

Question

Spring application is working fine when it runs locally on my machine and accesses Kafka through docker, but it doesn't work when I add my Spring Application as a container inside of the cluster. I get error message: "Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available"

Listed below is the docker-compose, dockerfile, and application.properties of the spring application.

docker-compose.yml

version: "3"

services:
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    ports:
      - '2181:2181'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: 'bitnami/kafka:latest'
    ports:
      - '9092:9092'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_LISTENERS=PLAINTEXT://:9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    depends_on:
      - zookeeper
  app:
    image: 'someuser/imagename'
    ports:
      - '8080:8080'
    depends_on:
      - kafka

applications.properties

server.port = 8080
spring.kafka.consumer.bootstrap.servers=localhost:9092
spring.kafka.consumer.group-id=mygroup 
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consume.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer

spring.kafka.producer.bootstrap-servers=localhost:9092
spring.kafka.producer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.producer.value-deserialzier=org.apache.kafka.common.serialization.StringDeserializer

Dockerfile

FROM openjdk:17

WORKDIR /app

EXPOSE 8080
EXPOSE 9092

COPY .mvn/ .mvn
COPY mvnw pom.xml ./
RUN ./mvnw dependency:go-offline

COPY src ./src

CMD ["./mvnw", "spring-boot:run"]

Answer 1

With spring.kafka.producer.bootstrap-servers=localhost:9092 and spring.kafka.consumer.bootstrap.servers=localhost:9092 your application is technically calling it's own container on port 9092 and not the one that kafka is deployed on.

If you change localhost:9092 to kafka:9092 it should work ( https://docs.docker.com/compose/networking/ ). (locally it wont, but it should connect when deployed through docker)

Try setting a different application profile for local and docker deployments.