How to get a Kafka Topic Lag in Java

Question

I want to see the lag position of a kafka topic in java. someone here says that below code will work.

AdminClient client = AdminClient.createSimplePlaintext("localhost:9092");
Map<TopicPartition, Object> offsets = JavaConversions.asJavaMap(
client.listGroupOffsets("groupID"));
Long offset = (Long) offsets.get(new TopicPartition("topic", 0));

But when I tried to import kafka.admin.AdminClient that listGroupOffsets method is not there. Please help me with this.

Answer 1

Method listGroupOffsets was introduced to AdminClient.scala starting 0.10.2. See KAFKA-3853 for details. So you should use Kafka 0.10.2.0 or upwards.

Answer 2

您可以使用https://github.com/yahoo/kafka-manager并可以使用其http Rest API获取消费者组延迟和其他详细信息。

Answer 3

I am using Spring framework. Using the below code, you can get the metrics via java.The code works.

@Component
public class Receiver {

private static final Logger LOGGER =
  LoggerFactory.getLogger(Receiver.class);


@Autowired
private KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry;

  public void testlag() {
  for (MessageListenerContainer messageListenerContainer : kafkaListenerEndpointRegistry
            .getListenerContainers()) {
      Map<String, Map<MetricName, ? extends Metric>> metrics = messageListenerContainer.metrics();
      metrics.forEach( (clientid, metricMap) ->{
          System.out.println("------------------------For client id : "+clientid);
          metricMap.forEach((metricName,metricValue)->{
              //if(metricName.name().contains("lag"))
              System.out.println("------------Metric name: "+metricName.name()+"-----------Metric value: "+metricValue.metricValue());
          });
      });
        }
  }