The requirement is to count the number of messages in a Kafka topic using Scala or Spark programming. I am new to both the programming so i am not sure how can it be done. Can anybody help me with the code or guide me how can it be achieved.
The methods for a given org.apache.kafka.clients.consumer.KafkaConsumer
are: beginningOffsets(Collection<TopicPartition>)
and endOffsets(Collection<TopicPartition>)
.
You need to remember that messages can be removed by kafka broker, that's why there is a need for beginning ones.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.