Python-Kafka：程序以交互模式而不是脚本模式运行

Question

I am trying to send some messages to a Kafka topic in Machine_2 through a python script in Machine_1 . Both Machine_2 and Machine_1 are in the same network and both are VM's in Azure.

Code: sampl.py

from kafka import KafkaProducer
Producer = KafkaProducer(bootstrap_servers=['Machine_2:9092'])
Producer.send('test', 'hello')

If I run the above code as

python sampl.py

There is no messages reaching to the Machine_2 . However, If I do:

python -i sampl.py

Then the messages reach to the Machine_2 . I checked the same using kafka-console-consumer.sh . I did yum update in Machine_1 thinking there might be some libraries missing here. But no luck yet.

Thanks.

Answer 1

kafka-python maintainer here. Producer.send('test', b'hello') is asynchronous and does not provide immediate delivery. What you are likely seeing is that the python interpreter is shutting down before the producer has a chance to complete the network send.

If you want to wait until the message is sent before finishing the script, you should use .get(timeout=...). So try:

Producer.send('test', b'hello').get(timeout=1000)

Or alternately, you can call flush() to do the same for all unsent messages:

Producer.flush(timeout=1000)