[英]Kafka producer future get timeout
I'm trying to handle failures in case the producer fails to send message to Kafka:我正在尝试处理失败,以防生产者无法向 Kafka 发送消息:
try {
Future<RecordMetadata> res = producer.send(new ProducerRecord<>(topic, msg.key(), msg));
log.info("Waiting for confirmation from kafka for message : \n {}",msg.toString());
record = res.get(30,TimeUnit.SECONDS);
log.info("Successfully produced message : msg")
} catch (InterruptedException | ExecutionException | TimeoutException e) {
log.error("The following message wasnt sent to kafka because of an error : {}", msg.toString(), e);
}
When I try to produce a message to a topic that doesn't exist I'm seeing the following error:当我尝试向不存在的主题生成消息时,我看到以下错误:
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Topic just_fake_topic not present in metadata after 60000 ms.
at org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:1307) ~[kafka-clients-2.5.0.jar!/:na]
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:962) ~[kafka-clients-2.5.0.jar!/:na]
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:862) ~[kafka-clients-2.5.0.jar!/:na]
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:750) ~[kafka-clients-2.5.0.jar!/:na]
at com.xx.xx.produceMessage.handleMessage(produceMessage.java:27) ~[classes!/:0.0.1-SNAPSHOT]
at
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: org.apache.kafka.common.errors.TimeoutException: Topic just_fake_topic not present in metadata after 60000 ms.
I'm getting the timeout after 60 seconds not 30 seconds as I configured in the get method of the future object.正如我在未来 object 的 get 方法中配置的那样,我在 60 秒后得到超时,而不是 30 秒。
I tried also to configure metadata.max.idle.ms=30000
and max.block.ms=30000
in my kafka.properties because I was also getting the following warning for about 5 minutes but it didn't help:我还尝试在我的 kafka.properties 中配置metadata.max.idle.ms=30000
和max.block.ms=30000
因为我也收到了大约 5 分钟的以下警告,但它没有帮助:
2020-06-28 14:31:26.009 WARN 10291 --- [ad | producer-1] org.apache.kafka.clients.NetworkClient : [Producer clientId=producer-1] Error while fetching metadata with correlation id 695 : {just_fake_topic =UNKNOWN_TOPIC_OR_PARTITION}
Any idea why?知道为什么吗?
The exception might be getting thrown from the Kafka network thread.异常可能是从 Kafka 网络线程中抛出的。
https://jaceklaskowski.gitbooks.io/apache-kafka/content/kafka-producer-internals-Sender.html https://jaceklaskowski.gitbooks.io/apache-kafka/content/kafka-producer-internals-Sender.html
Maybe provide a callback as a second parameter for send.也许提供一个回调作为发送的第二个参数。 Note: callback will run on the kafka network thread.注意:回调将在 kafka 网络线程上运行。
Kafka Producer: Handle Exception in Async Send with Callback Kafka Producer:使用回调处理异步发送中的异常
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.