[英]What is negative effects of setting max.poll.interval.ms larger than request.timeout.ms in Kafka consumer configs
According to Kafka documentation;根据 Kafka 文档;
The new Java Consumer now supports heartbeating from a background thread.
新的 Java Consumer 现在支持来自后台线程的心跳。 There is a new configuration max.poll.interval.ms which controls the maximum time between poll invocations before the consumer will proactively leave the group (5 minutes by default).
有一个新的配置 max.poll.interval.ms 控制消费者主动离开组之前轮询调用之间的最长时间(默认为 5 分钟)。 The value of the configuration request.timeout.ms must always be larger than max.poll.interval.ms because this is the maximum time that a JoinGroup request can block on the server while the consumer is rebalancing, so we have changed its default value to just above 5 minutes.
配置 request.timeout.ms 的值必须始终大于 max.poll.interval.ms 因为这是消费者重新平衡时 JoinGroup 请求可以在服务器上阻塞的最长时间,因此我们更改了其默认值到 5 分钟以上。
But I couldn't understand the possible results of making max.poll.interval.ms larger than request.timeout.ms.但我无法理解使 max.poll.interval.ms 大于 request.timeout.ms 的可能结果。
Can anybody explain this part of document:任何人都可以解释这部分文件:
because this is the maximum time that a JoinGroup request can block on the server while the consumer is rebalancing
因为这是消费者重新平衡时 JoinGroup 请求可以在服务器上阻塞的最长时间
Now I'am using these consumer parameters in my project:现在我在我的项目中使用这些消费者参数:
kafka.consumer.session.timeout.ms=30000
kafka.consumer.heartbeat.interval.ms=10000
kafka.consumer.request.timeout.ms=31000
kafka.consumer.max.poll.interval.ms=259200000
kafka.consumer.max.partition.fetch.bytes=10242880
What would be the negative effects of these usage?这些用法会产生什么负面影响?
Note: My kafka-clients version is 1.1.0注意:我的 kafka-clients 版本是 1.1.0
From Kafka version 2.0.0 this rule is not valid anymore.从 Kafka 2.0.0 版开始,此规则不再有效。 From Kafka docs Notable changes in 2.0.0:
来自 Kafka 文档 2.0.0 中的显着变化:
Also as part of KIP-266, the default value of request.timeout.ms has been changed to 30 seconds.
同样作为 KIP-266 的一部分,request.timeout.ms 的默认值已更改为 30 秒。 The previous value was a little higher than 5 minutes to account for maximum time that a rebalance would take.
先前的值略高于 5 分钟,以考虑重新平衡所需的最长时间。 Now we treat the JoinGroup request in the rebalance as a special case and use a value derived from max.poll.interval.ms for the request timeout.
现在我们将重新平衡中的 JoinGroup 请求视为一种特殊情况,并使用从 max.poll.interval.ms 派生的值作为请求超时。 All other request types use the timeout defined by request.timeout.ms
所有其他请求类型使用 request.timeout.ms 定义的超时
request.timeout.ms
specifies how long the consumer will wait for a response from the broker. request.timeout.ms
指定消费者等待代理响应的时间。
max.poll.interval.ms
is used in a couple of use cases. max.poll.interval.ms
用于几个用例。 It first specifies how often the consumer was to intereact with brokers (to confirm it's alive) but it's also used when the consumer joins a group.它首先指定消费者与代理交互的频率(以确认它是活着的),但在消费者加入组时也会使用它。 In this case, it indicates how long the broker can take to respond to the join group request.
在这种情况下,它指示代理可以花费多长时间来响应加入组请求。
So if max.poll.interval.ms
is larger than request.timeout.ms
, you see that the broker may take longer to respond than the consumer will wait.因此,如果
max.poll.interval.ms
大于request.timeout.ms
,您会看到代理的响应时间可能比消费者等待的时间更长。 So the consumer could timeout requests.所以消费者可以超时请求。
In many cases, the defaults values for these 2 configurations are relatively good.在很多情况下,这两种配置的默认值都比较好。 Instead of overriding them to the large values I see in your question, you should first start by clearly identify how you want to change the default behaviour.
不要将它们覆盖为我在您的问题中看到的大值,您应该首先清楚地确定您想要如何更改默认行为。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.