在不同的服务器上为 Kafka 集群运行 provectus/kafka-ui

Question

I'm in a bit of trouble trying to run provectus/kafka-ui docker containers. I have a collection of servers all running on the same internal.network. On three of these servers a Kafka cluster is implemented (Zookeeper and Broker). The structure of the Broker looks like the following:

192.168.0.237:9092
192.168.0.7:9092
192.168.0.54:9092

The problem is that there is no Docker instance running on any of these servers. What I would like to achieve now is to install provectus/kafka-ui on another server (192.168.0.8) and make it accessible from within the.network via a specific port (eg 8080).

I have found a few docker-compose.yml examples https://github.com/provectus/kafka-ui . However, the Kafka cluster is created on the same server as kafka-ui. In my case, the cluster already exists and only kafka-ui is missing on another server. Can anyone tell me how I can adapt the docker-compose.yml so that kafka-ui can be installed on 192.168.0.8 and made available from outside via port 8080?

Kind regards and thanks in advance

version: '3'
services: 
  kafka-ui:
    container_name: kafka-ui
    image: provectuslabs/kafka-ui:latest
    ports:
      - 8080:8080
    depends_on:
      - zookeeper
      - kafka1
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka1:19091
      KAFKA_CLUSTERS_0_ZOOKEEPER: zookeeper:2181
      KAFKA_CLUSTERS_0_JMXPORT: 9997
      SERVER_SERVLET_CONTEXT_PATH: /kafkaui
      AUTH_TYPE: "LOGIN_FORM"
      SPRING_SECURITY_USER_NAME: test_user
      SPRING_SECURITY_USER_PASSWORD: test
  zookeeper: 
    image: confluentinc/cp-zookeeper:5.2.4
    hostname: zookeeper
    ports: 
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
    volumes: 
      - ./data/zookeeper/data:/data
      - ./data/zookeeper/datalog:/datalog
  kafka1:
    image: confluentinc/cp-kafka:5.3.1
    hostname: kafka1
    ports: 
      - "9091:9091"
    environment: 
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka1:19091,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9091
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
      KAFKA_BROKER_ID: 1
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 2
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "False"
    volumes: 
      - ./data/kafka1/data:/var/lib/kafka/data
    depends_on: 
      - zookeeper
  kafka2:
    image: confluentinc/cp-kafka:5.3.1
    hostname: kafka2
    ports: 
      - "9092:9092"
    environment: 
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka2:19092,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 2
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 2
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "False"
    volumes: 
      - ./data/kafka2/data:/var/lib/kafka/data
    depends_on: 
      - zookeeper 
  kafka3:
    image: confluentinc/cp-kafka:5.3.1
    hostname: kafka3
    ports: 
      - "9093:9093"
    environment: 
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka3:19093,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
      KAFKA_BROKER_ID: 3
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 2
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "False"
    volumes: 
      - ./data/kafka3/data:/var/lib/kafka/data
    depends_on: 
      - zookeeper
  kafdrop: 
    image: obsidiandynamics/kafdrop
    restart: "no"
    ports: 
      - "9000:9000"
    environment: 
      KAFKA_BROKERCONNECT: "kafka1:19091"
    depends_on: 
      - kafka1
      - kafka2
      - kafka3

Answer 1

You don't need Docker on the Kafka brokers... Also, IP doesn't really matter as long as advertised.listeners on Kafka is setup correctly.

Remove the three Kafka containers, the Zookeeper container, and simply change these to your actual Kafka brokers + Zookeeper servers

KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS
KAFKA_CLUSTERS_0_ZOOKEEPER

Also remove kafdrop? Since it's basically another UI with overlapping features.

---

Or, install pnpm and Java, and you can run the UI + API without Docker , just like you're doing with Kafka.

https://github.com/provectus/kafka-ui/tree/master/kafka-ui-react-app