Unable to get number of messages in kafka topic

Question

I am fairly new to kafka. I have created a sample producer and consumer in java. Using the producer, I was able to send data to a kafka topic but I am not able to get the number of records in the topic using the following consumer code.

    public class ConsumerTests {
        public static void main(String[] args) throws Exception {
            BasicConfigurator.configure();

            String topicName = "MobileData";
            String groupId = "TestGroup";
            Properties properties = new Properties();
            properties.put("bootstrap.servers", "localhost:9092");
            properties.put("group.id", groupId);
            properties.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
            properties.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

            KafkaConsumer<String, String> kafkaConsumer = new KafkaConsumer<>(properties);
            kafkaConsumer.subscribe(Arrays.asList(topicName));


try {
  while (true) {
    ConsumerRecords<String, String> consumerRecords = consumer.poll(100);
    System.out.println("Record count is " + records.count());
  }
} catch (WakeupException e) {
  // ignore for shutdown
} finally {
  consumer.close();
}

}
}

I don't get any exception in the console but consumerRecords.count() always returns 0, even if there are messages in the topic. Please let me know, if I am missing something to get the record details.

Answer 1

The poll(...) call should normally be in a loop. It's always possible for the initial poll(...) to return no data (depending on the timeout) while the partition assignment is in progress. Here's an example:

try {
  while (true) {
    ConsumerRecords<String, String> records = consumer.poll(100);
    System.out.println("Record count is " + records.count());
  }
} catch (WakeupException e) {
  // ignore for shutdown
} finally {
  consumer.close();
}

For more info see this relevant article :