Log offset, partition and topic from Kafka message

Question

I want to log or print offset, partition, and topic from Kafka's message. I can print the message value but I want to see which offset and partition from Kafka using python so that I can debug my code

from kafka import KafkaConsumer

consumer = KafkaConsumer('my-topic',
                         group_id='my-group',
                         bootstrap_servers=['localhost:9092'])
for message in consumer:
    print(f"Message is {message.value()}") 

Answer 1

That data should be available using dot notation

The consumer iterator returns ConsumerRecords, which are simple namedtuples that expose basic message attributes: topic, partition, offset , key, and value:

https://github.com/dpkp/kafka-python

Answer 2

from kafka import KafkaConsumer

consumer = KafkaConsumer('my-topic',
                         group_id='my-group',
                         bootstrap_servers=['localhost:9092'])
for message in consumer:
    logger.debug(f"Consumer data: offset-{message.offset()} partition-{message.partition()} topic-{message.topic()}")
    print(f"Message is {message.value()}")