Kafka：此服务器不是该主题分区的领导者

Question

Possible duplicate of Kafka - This server is not the leader for that topic-partition but there is no accepted answer nor a clear solution.

I have a simple java program to produce the message to Kafka:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("acks", "all");
props.put("retries", 1);
props.put("batch.size", 16384);
props.put("linger.ms", 100);
props.put("buffer.memory", 33554432);
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "com.company.project.KafkaValueSerializer");
MyMessage message = new MyMessage();
Producer<String, MyMessage> producer = new KafkaProducer<>(props);
Future<RecordMetadata> future =
    producer.send(new ProducerRecord<String, MyMessage>("My_Topic", "", message));

I am getting the following exception:

Exception in thread "main" java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition.
    at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:94)
    at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:64)
    at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:29)
    at 
Caused by: org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition.

When I tried with kafka-console-producer , I am getting the following error:

D:\kafka_2.11-0.10.2.0\bin\windows>kafka-console-producer.bat --broker-list localhost:9092  --topic My_Topic
hello
[2018-10-25 17:05:27,225] WARN Got error produce response with correlation id 3 on topic-partition My_Topic-0, retrying (2 attempts left). Error: NOT_LEADER_FOR_PARTITION (org.apache.kafka.clients.producer.internals.Sender)
[2018-10-25 17:05:27,335] WARN Got error produce response with correlation id 5 on topic-partition My_Topic-0, retrying (1 attempts left). Error: NOT_LEADER_FOR_PARTITION (org.apache.kafka.clients.producer.internals.Sender)
[2018-10-25 17:05:27,444] WARN Got error produce response with correlation id 7 on topic-partition My_Topic-0, retrying (0 attempts left). Error: NOT_LEADER_FOR_PARTITION (org.apache.kafka.clients.producer.internals.Sender)
[2018-10-25 17:05:27,544] ERROR Error when sending message to topic My_Topic with key: null, value: 5 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition.

When I describe my topic, I have the following information:

Topic:My_Topic        PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: My_Topic       Partition: 0    Leader: 0 Replicas: 0     Isr: 0

I tried creating a new topic and producing messages as mentioned in Quick start guide then the above steps work well.

What correction should I make in My_Topic or producer configuration so that I can publish messages successfully in this topic?

Answer 1

Had it been the case of "console client working, but Java program not working" , then the solution of 'changing retry limit' may have helped.

As both the Java program and the built-in command-line producer are not able to connect to Kafka, I suspect that the problem could be due to stale configuration.

(Example: topic deleted and created again with the same name, but with different partition count).

Deleting the old log files of zookeeper and Kafka and creating the topic again, or creating a topic with a different name will resolve the issue.