来自 python NoBrokersAvailable 的 Kafka Consumer

Question

I am using docker to run a kafka producer with the command

kafka-console-producer.sh --topic USER_CREATED_TOPIC --broker-list xxx.xx.x.x:9092`

where the x are numbers from the assigned broker ip.

My server.properties file contains

advertised.port=9092
advertised.host.name=xxx.xx.x.x.
listeners=PLAINTEXT://xxx.xx.x.x:9092 line
advertised.listeners=PLAINTEXT://xxx.xx.x.x:9092

Whenever i am starting a consumer from the docker container with the command

kafka-console-consumer.sh --topic USER_CREATED_TOPIC --from-beginning --bootstrap-server xxx.xx.x.x:9092

and write something in my producer console i get the result in the consumer.(no error here)

However, when i try to connect via python script using:

from kafka import KafkaConsumer
  consumer = 
  kafkaConsumer("USER_CREATED_TOPIC",bootstrap_servers= 
  ['xxx.xx.x.x:9092'])
for msg in consumer:
     print (msg)

I am getting an NoBrokersAvailable error.

I read couple of threads on stackoverflow ( listed the added items on the server.properties based on those answers) but i am still unable to connect to the kafka producer via python.

Any help is appreciated.

Answer 1

The only syntaxical issue I see is that the server address should not be in a list, so like this:

from kafka import KafkaConsumer
consumer = KafkaConsumer('sample', bootstrap_servers='0.0.0.0:9092')
for message in consumer:
    print(message)

No running cluster would also give that error. First create a topic:

kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic sample

Then test if it exists:

bin/kafka-topics.sh --list --zookeeper localhost:2181
# sample

Lastly make sure there is a producer creating messages to listen to:

from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers='0.0.0.0:9092')
producer.send('sample', b'Hello, World!')
producer.send('sample', key=b'message-two', value=b'This is Kafka-Python')