如何在 Kafka 中访问消费者的特定指标

Question

i am working with a java producer and consumer, i am trying to visualize my consumer lag (i have it lagging in purpose) from my java application. I want to log that lag as a stdout. I have tried to use JMX but i havent been able to use it in the java application. I have used this (just a snippet, the set up of the consumer is done in the application)

private KafkaConsumer<String, String> consumer

Map<MetricName, ? extends Metric> metrics = consumer.metrics();
System.out.println(metrics);

But it only gives me the description of the metrics but not the value. I see in my Kafka CLI and it shows the right lag for that consumer.

Answer 1

First identify which metrics you are interested in using the docs or printing all metrics like in your question.

Then find the metrics from the Consumer. For example, if you are interested in request-latency-avg :

// Find the metrics you are interested in
Metric requestLatencyAvgMetric = null;
for (Entry<MetricName, ? extends Metric> entry : consumer.metrics().entrySet()) {
    if ("request-latency-avg".equals(entry.getKey().name())) {
        requestLatencyAvgMetric = entry.getValue();
    }
}

Then you can retrieve the value when you need it using:

requestLatencyAvgMetric.metricValue()