[英]I have a Topic with 0 leader and 0 isr: UNKNOWN_TOPIC_OR_PARTITION
[英]Kafka on Kubernetes - UNKNOWN_TOPIC_OR_PARTITION and LEADER_NOT_AVAILABLE error
这是一个后续问题本 。 我设法做到以下几点:
我使用zkCli创建了一个主题test-topic
,并验证了它是否已创建。 之后,我开始了Kafka控制台制作人。
.\kafka-console-producer.bat --broker-list EXTERNAL_IP_1:9093,EXTERNAL_IP_2:9093,EXTERNAL_IP_3:9093,EXTERNAL_IP_4:9093,EXTERNAL_IP_5:9093 --topic test-topic --property parse.key=true --property key.
separator=:
>afkjdshasdkfjhsdkjsf:128379127893123
>[2018-05-09 17:35:51,622] WARN [Producer clientId=console-producer] Got error produce response with correlation id 9 on topic-partition test-topic-0, retrying (2 attempts left). Error: UNKNOWN_TOPIC_OR_PARTITION (org.apache.kafka.clients.producer.internals.Sender)
[2018-05-09 17:35:51,623] WARN [Producer clientId=console-producer] Received unknown topic or partition error in produce request on partition test-topic-0. The topic/partition may not exist or the user may not have Describe access to it (org.apache.kafka.clients.producer.internals.Sender)
[2018-05-09 17:35:51,649] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 10 : {test-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2018-05-09 17:35:51,720] WARN [Producer clientId=console-producer] Got error produce response with correlation id 11 on topic-partition test-topic-0, retrying (1 attempts left). Error: UNKNOWN_TOPIC_OR_PARTITION (org.apache.kafka.clients.producer.internals.Sender)
[2018-05-09 17:35:51,720] WARN [Producer clientId=console-producer] Received unknown topic or partition error in produce request on partition test-topic-0. The topic/partition may not exist or the user may not have Describe access to it (org.apache.kafka.clients.producer.internals.Sender)
[2018-05-09 17:35:51,773] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 12 : {test-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2018-05-09 17:35:51,823] WARN [Producer clientId=console-producer] Got error produce response with correlation id 13 on topic-partition test-topic-0, retrying (0 attempts left). Error: UNKNOWN_TOPIC_OR_PARTITION (org.apache.kafka.clients.producer.internals.Sender)
[2018-05-09 17:35:51,823] WARN [Producer clientId=console-producer] Received unknown topic or partition error in produce request on partition test-topic-0. The topic/partition may not exist or the user may not have Describe access to it (org.apache.kafka.clients.producer.internals.Sender)
[2018-05-09 17:35:51,913] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 14 : {test-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2018-05-09 17:35:51,936] ERROR Error when sending message to topic test-topic with key: 20 bytes, value: 15 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition.
[2018-05-09 17:35:51,945] WARN [Producer clientId=console-producer] Received unknown topic or partition error in produce request on partition test-topic-0. The topic/partition may not exist or the user may not have Describe access to it (org.apache.kafka.clients.producer.internals.Sender)
[2018-05-09 17:35:52,034] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 16 : {test-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2018-05-09 17:35:52,161] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 20 : {test-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2018-05-09 17:40:52,288] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 25 : {test-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
根据Zookeeper的说法,我的Kafka经纪人“ kafka-2”是该主题的领导者:
get /kafka/brokers/topics/test-topic/partitions/0/state
{"controller_epoch":5,"leader":2,"version":1,"leader_epoch":0,"isr":[2,1]}
但是pod kafka-2在Log中抛出错误
[2018-05-09 15:21:02,524] ERROR [ReplicaFetcherThread-0-2], Error for partition [test-topic,0] to broker 2:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
我不太确定为什么会这样,对我来说配置看起来不错。 使我的Kafka集群在Kubernetes上运行还缺少什么呢?
请注意,我还尝试完全擦除群集(缩小kafka群集,删除kafka存储,缩小zk群集,删除zk存储,放大zk,放大kafka),但无济于事。
我已经修复了。 问题是我的无头服务既包含内部端口,也包含外部端口。
现在,我的无头服务仅包含内部端口:
apiVersion: v1
kind: Service
metadata:
name: kafka-hs
labels:
app: kafka
spec:
ports:
- port: 29092
name: server
clusterIP: None
selector:
app: kafka
我的公开外部ip的按服务显示的服务包含外部端口(请注意,RedHat OpenShift脚本处理这些服务的外部ip分配,服务定义中未涉及):
apiVersion: v1
kind: Service
metadata:
name: kafka-es-4
labels:
app: kafka
namespace: whatever
spec:
ports:
- port: 9093
name: kafka-port
protocol: TCP
selector:
statefulset.kubernetes.io/pod-name: kafka-4
app: kafka
type: LoadBalancer
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.