如何在 Apache Beam 中使用 KafkaIO 指定 kafka 代理

Question

I'm trying to setup a KafkaIO pipeline but i can't figure out how to specify brokers. Speficying brokername and port doesn't seem to do it. At no point am I specifying where my kafka cluster is:

       pipeline
            .apply(KafkaIO.<Long, String>read()
                    .withBootstrapServers("broker.id__1:9092")
                    .withTopic("pageviews")

which gives me the error:

        Exception in thread "main" org.apache.beam.sdk.Pipeline$PipelineExecutionException: org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
        at org.apache.beam.sdk.Pipeline.run(Pipeline.java:323)
...
    Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers

My Kafka cluster is running locally and the broker id is correct.

What am i missing? What's the right way of specifying brokers?

Answer 1

Turns out it's simply

withBootstrapServers("localhost:9092")