How to change target of the Spring Cloud Stream Kafka binder?

Question

Using Spring cloud Stream 2.1.4 with Spring Boot 2.1.10, I'm trying to target a local instance of Kafka. This is an extract of my projetc configuation so far:

spring.kafka.bootstrap-servers=PLAINTEXT://localhost:9092
spring.kafka.streams.bootstrap-servers=PLAINTEXT://localhost:9092
spring.cloud.stream.kafka.binder.brokers=PLAINTEXT://localhost:9092
spring.cloud.stream.kafka.binder.zkNodes=localhost:2181
spring.cloud.stream.kafka.streams.binder.brokers=PLAINTEXT://localhost:9092
spring.cloud.stream.kafka.streams.binder.zkNodes=localhost:2181

But the binder keeps on calling a wrong target:

java.io.IOException: Can't resolve address: kafka.example.com:9092

How can can I specify the target if those properties won't do he trick? More, I deploy the Kafka instance through a Docker Bitnami image and I'd prefer not to use SSL configuration (see PLAINTEXT protocol) but I'm don't find properties for basic credentials login. Does anyone know if this is hopeless?

This is my docker-compose.yml

version: '3'
services: 
  zookeeper:
    image: bitnami/zookeeper:latest
    container_name: zookeeper
    environment:
      - ZOO_ENABLE_AUTH=yes
      - ZOO_SERVER_USERS=kafka
      - ZOO_SERVER_PASSWORDS=kafka_password
    networks:
      - kafka-net
  kafka:
    image: bitnami/kafka:latest
    container_name: kafka
    hostname: kafka.example.com
    depends_on:
      - zookeeper
    ports:
      - 9092:9092
    environment:
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://:9092
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_ZOOKEEPER_USER=kafka
      - KAFKA_ZOOKEEPER_PASSWORD=kafka_password
    networks:
      - kafka-net
networks:
  kafka-net:
    driver: bridge

Thanks in advance

Answer 1

The hostname isn't the issue, rahter the advertised listeners protocol//:port mapping that causes the hostname to be advertised, by default. You should change that, rather than the hostname.

  kafka:
    image: bitnami/kafka:latest
    container_name: kafka
    hostname: kafka.example.com # <--- Here's what you are getting in the request
    ...
    environment:
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://:9092 # <--- This returns the hostname to the clients

If you plan on running your code outside of another container, you should advertise localhost in addition to, or instead of the container hostname.

One year later, my comment still is not been merged into the bitnami README , where I was able to get it working with the following vars (changed to match your deployment)

KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_CFG_LISTENERS=PLAINTEXT://:29092,PLAINTEXT_HOST://:9092
KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka.example.com:29092,PLAINTEXT_HOST://localhost:9092

Answer 2

All right: got this to work by looking twice to the "dockerfile" (thx to cricket_007):

kafka:
...
    hostname: localhost

For the record: I could get rid of all properties above, default being for Kafka localhost:9092