卡夫卡消费者不显示消息

Question

I am trying to understand how a Kafka producer works. Below is the python producer code I wrote to send a message. I started the Kafka console consumer first and then I am running the python code

from confluent_kafka import Producer
from Product import Product
from faker import Faker



if __name__ == '__main__':


    config = {
        "bootstrap.servers":"localhost:9092"
    }
    producer = Producer(config)
    fake = Faker()
    product = Product(fake.name())
    print(product.serial())
    producer.produce(topic="first_topic",value=product.serial())

The issue that I facing is that if I call flush method after calling produce the message appears on the console consumer however without flush the message doesn't appear in the console consumer. As per Kafka documentation flush will make the producer synchronous. Is there a way to avoid using the flush and still make sure the message gets consumed. Thank you

Answer 1

您要么需要刷新，要么需要通过更多消息填充生产者的批量大小，或者通过 confluent_kafka 包中的一些 librdkafka 配置设置将批量大小减少到单个消息以下

Answer 2

您可以按照@OneCricteer 的建议填充批量大小，也可以将 linger.ms 设置为 0。这样它就不会在发送前等待填充到批量大小。