如何从 cp-enterprise-kafka:6.2.1 迁移到 cp-kafka:6.2.1 docker 镜像？ 对我来说失败了

Question

When I change the image in docker-compose.yml from enterprise cp-enterprise-kafka to regular cp-kafka

it fails to start with:

My config in docker-compose.yml is:

version: '3'

services:
  zookeeper-1:
    image: confluentinc/cp-zookeeper:6.2.1
    hostname: zookeeper-1
    container_name: zookeeper-1
    ports:
      - "2181:2181"
      - "2888:2888"
      - "3888:3888"
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    volumes:
      - ./kafka-data/zookeeper-1:/var/lib/zookeeper/data
      - ./kafka-data/zookeeper-logs-1:/var/lib/zookeeper/log
    networks:
      - mynet

  broker-1:
    image: confluentinc/cp-enterprise-kafka:6.2.1
    hostname: broker-1
    container_name: broker-1
    depends_on:
      - zookeeper-1
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: '192.168.100.14:2181'

      KAFKA_LISTENERS: OUTSIDE://0.0.0.0:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: OUTSIDE:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: OUTSIDE://192.168.100.14:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: OUTSIDE

      KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: 'LogAppendTime'

      KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_CONNECTIONS_MAX_IDLE_MS: 31536000000 # 1 year

      CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: 192.168.100.14:9092
      CONFLUENT_METRICS_REPORTER_ZOOKEEPER_CONNECT: 192.168.100.14:2181
      CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
      CONFLUENT_METRICS_ENABLE: 'true'
      CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous'
    volumes:
      - ./kafka-data/kafka-1:/var/lib/kafka/data
    networks:
      - mynet

networks:
  mynet:
    driver: bridge

ie I change only broker-1 image from cp-enterprise-kafka to cp-kafka.

I noticed that controllerEpoch is much lesser than brokerEpoch - maybe this is a reason?

Any ideas? Thanks!

Answer 1

RESOLVED (but don't know how exactly):

1. stopped zookeeper and broker
2. copy all zookeeper data (backup)
3. start zookeeper and broker and got this (wrong cluster):

broker-1       | [2021-10-19 09:25:25,648] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
broker-1       | kafka.common.InconsistentClusterIdException: The Cluster ID MlRI76IVQ8y4nTZgigyUGw doesn't match stored clusterId Some(HXaFk1peQcaR23TAqh0Z2g) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong.
broker-1       |        at kafka.server.KafkaServer.startup(KafkaServer.scala:218)
broker-1       |        at kafka.Kafka$.main(Kafka.scala:109)
broker-1       |        at kafka.Kafka.main(Kafka.scala)

4. stopped zookeeper and broker
5. copied (backed up previously) zookeeper data back
6. started... and all starts ok!

Maybe even zookeeper data backup not needed at all and it's fine just to change broker's cluster.id inside /var/lib/kafka/data/meta.properties