Kafka API - 如果 bootstrap.servers 属性设置为一系列已关闭但同一集群中有其他代理启动的代理，会发生什么情况？

Question

Imagine this situation on a Producer application.

final Properties streamsConfiguration = new Properties();
streamsConfiguration.put(StreamsConfig.APPLICATION_ID_CONFIG, "my-app");
streamsConfiguration.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "broker1:9092,broker2:9092");

As you can see, there's 2 Kafka Brokers.

Now, what will happen if:

1. In a point in time in the future I have to scale up my Kafka cluster by adding 2 more brokers (broker3 and broker4).
2. In other point in time in the future, broker1 and broker2 goes down by a technical issue.

broker3 and broker4 still up and working. As they're running in Zookeeper, one of them will be set as a leader, but my application doesn't know broker3 or broker4.

What will happen? Will my application stop working? If yes, I suppose there's a solution for that. What would be?

Answer 1

According to the documentation on the bootstrap.servers in the Producer Configs it is sufficient to have at least one running broker listed in the list. The one connection will be able to establish connection to the entire Kafka cluster.

Of course, if you are listing only brokers that are not existing/down for some reason, then your application would fail.

Here is the part of the documentation:

A list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping—this list only impacts the initial hosts used to discover the full set of servers. This list should be in the form host1:port1,host2:port2,.... Since these servers are just used for the initial connection to discover the full cluster membership (which may change dynamically), this list need not contain the full set of servers (you may want more than one, though, in case a server is down).