python-kafka：消费者是否可以根据消息属性跳过消息？

Question

Given a set of messages, each with a priority attribute. Is it possible for a consumer to skip or ignore messages whose priority attribute is not high enough?

At first I deserialized messages consumed by the kafka-python consumer, checked the priority level and discarded those messages that did not meet the required priority level.

But I am concerned that if the message is large, considerable time will be wasted deserializing messages that will not be processed further. Is there not a way for this to be done upon reading from the kafka queue?

Answer 1

Kafka does currently not support custom metadata in this way so the only way for the consumer to filter out certain messages is to read the payload. However there is an improvement proposal under discussion which would address this kind of situation and might be worth following.