[英]Kafka Consumer - JMX Properties
I enabled JMX on the kafka brokers on port 8081. When I view the MBean properties in jConsole, I only see the following for kafka.consumer
- 我在端口8081的kafka代理上启用了JMX。当我在jConsole中查看MBean属性时,仅对kafka.consumer看到以下
kafka.consumer
-
kafka.consumer:type=FetchRequestAndResponseMetrics,name=FetchRequestRateAndTimeMs,clientId=ReplicaFetcherThread-2-413
kafka.consumer:type=FetchRequestAndResponseMetrics,name=FetchResponseSize,clientId=ReplicaFetcherThread-0-413
But none of the other ones that are identified in here under Kafka Consumer Metrics are emitted by JMX. 但是,JMX不会发出在“卡夫卡消费者指标”下在此标识的其他参数。
Kafka Version # 0.8.2.1 Kafka版本#0.8.2.1
I am specifically interested in - kafka.consumer:type=ConsumerFetcherManager,name=MaxLag,clientId=([-.\\w]+)
我特别感兴趣
kafka.consumer:type=ConsumerFetcherManager,name=MaxLag,clientId=([-.\\w]+)
Any thoughts? 有什么想法吗?
The JMX PORT you are listening is the broker port. 您正在监听的JMX PORT是代理端口。 But the Mbean of kafka.consumer: is the consumer jvm metrics.
但是kafka.consumer:的Mbean是使用者jvm指标。 So if you have another JVM that is consume a topic, you can see kafka.consumer Mbeans.
因此,如果您有另一个使用主题的JVM,则可以看到kafka.consumer Mbeans。
ConsumerLag is an overloaded term in Kafka , it can stand for: ConsumerLag是Kafka中的重载术语 ,可以表示:
Consumer's metric : Calculated difference between a consumer's current log offset and a producer's current log offset. 消费者度量 :消费者当前对数偏移量与生产者当前对数偏移量之间的计算差。 You can find it under JMX bean, if you're using Java/Scala based consumer (eg pykafka consumer doesn't export metrics ):
如果您使用的是基于Java / Scala的使用者,则可以在JMX bean下找到它(例如pykafka使用者不导出指标 ):
kafka v0.8.2.x
: kafka
v0.8.2.x
:
kafka.consumer:type= ConsumerFetcherManager, name=MaxLag, clientId=([-.\\w]+)
kafka v0.9+
: kafka
v0.9+
:
kafka.consumer:type=consumer-fetch-manager-metrics,client-id=([-.w]+)
Consumer lag used to be stored in ZooKeeper (Kafka <= v0.8
), newer versions of Kafka have special topic __consumer_offsets
that stores each consumer's lag. 消费者滞后曾经存储在ZooKeeper中(Kafka
<= v0.8
),Kafka的较新版本具有特殊的主题__consumer_offsets
,用于存储每个消费者的滞后。 There are tools (eg kafka-manager ) that can compute lag by consuming messages from this topic and calculating lag. 有一些工具(例如kafka-manager )可以通过使用本主题中的消息并计算延迟来计算延迟。 In
kafka-manager
you have to enable this feature for each cluster: 在
kafka-manager
您必须为每个集群启用此功能:
[ ] Poll consumer information (Not recommended for large # of consumers)
[]轮询消费者信息(不建议大量消费者使用)
Broker's metric : Represent the offset differences between partition leaders and their followers. 经纪人指标 :代表分区领导者及其追随者之间的偏移差异。 You can find this metric under JMX bean:
您可以在JMX bean下找到此指标:
kafka.server:type=FetcherLagMetrics,name=ConsumerLag,clientId=([-.\\w]+),topic=([-.\\w]+),partition=([0-9]+)
This may help to find it for 0.8, but I am currently running a Kafka 0.10 broker and consumer. 这可能有助于找到0.8,但我目前正在运行Kafka 0.10经纪人和使用者。 When using a console consumer, I pointed
jconsole
to this consumer and found on the MBeans TAB: kafka.consumer
-> consumer-fetcher-manager-metric
-> consumer-1
-> Attributes
-> records-max-lag
. 使用控制台使用者时,我将
jconsole
指向该使用者,并在MBeans TAB上找到: kafka.consumer
> consumer-fetcher-manager-metric
kafka.consumer
consumer-fetcher-manager-metric
kafka.consumer
> consumer-1
> Attributes
> records-max-lag
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.