Kafka 消费者滞后指标

Question

We are using kafka 2.12.XXXX - and need to observe how much lag consumers have on all the partitions of a topic.
Is the the summation of the LAG column in ./kafka-consumer-groups.sh --describe an accurate representation of that

Answer 1

Yes, using the kafka-consumer-groups.sh script, you can get the consumer lag information for a given consumer group. The summation of the values under LAG column would be the total lag for that particular consumer group.

➜  ~ sh $KAFKA_HOME/bin/kafka-consumer-groups -bootstrap-server localhost:9092 --describe --group test-group
Consumer group 'test-group' has no active members.

TOPIC                PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID     HOST            CLIENT-ID
test-topic           0          53              53              0               -               -               -
test-topic           1          53              53              0               -               -               -

However, I would suggest using a command line tool like kt ( source ), as it is faster and can provide you a json output, which would be easier to work with.

➜  ~ kt group -brokers localhost:9092 -topic test-topic -group test-group
found 1 brokers
found 1 groups
found 1 topics
found partitions=[0 1] for topic=test-topic
{
  "name": "test-group",
  "topic": "test-topic",
  "offsets": [
    {
      "partition": 0,
      "offset": 53,
      "lag": 0
    },
    {
      "partition": 1,
      "offset": 53,
      "lag": 0
    }
  ]
}

The lag key in the output is the consumer lag for every partition for the given consumer group.