Google Cloud Kafka实例和Spring Boot Appengine应用连接错误：超时：提取主题元数据时超时

Question

我正在使用Google Cloud Kafka实例连接到云Appengine中的Spring启动服务。

但是在启动时，我得到了TimeoutException：提取主题元数据时超时到期

我在代码中使用以下配置：

application.yml：

kafka:
  bootstrap-servers: <private_ip of_my_kafka_instance>:9092

卡夫卡中的server.properties：

listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners=PLAINTEXT://<private_ip of_my_kafka_instance>:9092

ReceiverConfig.java

@Configuration
@EnableKafka
public class ReceiverConfig {

  @Value("${kafka.bootstrap-servers}")
  private String bootstrapServers;

  @Bean
  public Map<String, Object> consumerConfigs() {
    Map<String, Object> props = new HashMap<>();
    // list of host:port pairs used for establishing the initial connections to the Kafka cluster
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,
        bootstrapServers);
    props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
        StringDeserializer.class);
    props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
        StringDeserializer.class);
    // allows a pool of processes to divide the work of consuming and processing records
    props.put(ConsumerConfig.GROUP_ID_CONFIG, "helloworld");
    // automatically reset the offset to the earliest offset
    props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");

    return props;
  }

  @Bean
  public ConsumerFactory<String, String> consumerFactory() {
    return new DefaultKafkaConsumerFactory<>(consumerConfigs());
  }

  @Bean
  public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<String, String>> kafkaListenerContainerFactory() {
    ConcurrentKafkaListenerContainerFactory<String, String> factory =
        new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(consumerFactory());

    return factory;
  }

  @Bean
  public Receiver receiver() {
    return new Receiver();
  }
}

Receiver.java

public class Receiver {

    private final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    MessageService messageService;

    @KafkaListener(topics = "doMatch")
    public void receive(String payload) {
        boolean status = true;
        String flag = null;
        logger.info("received topic='doMatch' payload='{}'", payload);
        logger.info("Processing Message...");
    }
}

在应用程序上记录错误：

o.s.boot.SpringApplication               : Application run failed

org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'; nested exception is org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
    at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185) ~[spring-context-5.1.2.RELEASE.jar:5.1.2.RELEASE]
    at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53) ~[spring-context-5.1.2.RELEASE.jar:5.1.2.RELEASE]
    at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360) ~[spring-context-5.1.2.RELEASE.jar:5.1.2.RELEASE]
    at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158) ~[spring-context-5.1.2.RELEASE.jar:5.1.2.RELEASE]
    at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122) ~[spring-context-5.1.2.RELEASE.jar:5.1.2.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:879) ~[spring-context-5.1.2.RELEASE.jar:5.1.2.RELEASE]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.finishRefresh(ServletWebServerApplicationContext.java:161) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) ~[spring-context-5.1.2.RELEASE.jar:5.1.2.RELEASE]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) ~[spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
    at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.run(SpringBootServletInitializer.java:157) [spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
    at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:137) [spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
    at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:91) [spring-boot-2.1.0.RELEASE.jar:2.1.0.RELEASE]
    at org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:171) [spring-web-5.1.2.RELEASE.jar:5.1.2.RELEASE]
    at org.eclipse.jetty.plus.annotation.ContainerInitializer.callStartup(ContainerInitializer.java:140) [runtime-impl-third-party.jar:na]
    at org.eclipse.jetty.annotations.ServletContainerInitializersStarter.doStart(ServletContainerInitializersStarter.java:63) [runtime-impl-third-party.jar:na]
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [runtime-impl-third-party.jar:na]
    at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:330) [runtime-impl-third-party.jar:na]
    at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1406) [runtime-impl-third-party.jar:na]
    at com.google.apphosting.runtime.jetty9.AppEngineWebAppContext.startWebapp(AppEngineWebAppContext.java:171) [runtime-impl.jar:na]
    at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1368) [runtime-impl-third-party.jar:na]
    at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:778) [runtime-impl-third-party.jar:na]
    at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262) [runtime-impl-third-party.jar:na]
    at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:522) [runtime-impl-third-party.jar:na]
    at com.google.apphosting.runtime.jetty9.AppEngineWebAppContext.doStart(AppEngineWebAppContext.java:116) [runtime-impl.jar:na]
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) [runtime-impl-third-party.jar:na]
    at com.google.apphosting.runtime.jetty9.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:243) [runtime-impl.jar:na]
    at com.google.apphosting.runtime.jetty9.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:181) [runtime-impl.jar:na]
    at com.google.apphosting.runtime.jetty9.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:109) [runtime-impl.jar:na]
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchServletRequest(JavaRuntime.java:695) [runtime-impl.jar:na]
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchRequest(JavaRuntime.java:658) [runtime-impl.jar:na]
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:628) [runtime-impl.jar:na]
    at com.google.apphosting.runtime.JavaRuntime$NullSandboxRequestRunnable.run(JavaRuntime.java:820) [runtime-impl.jar:na]
    at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:269) [runtime-impl.jar:na]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171-google-v7]
Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata

有人可以帮忙吗... !!!

Answer 1

您需要在代理上正确配置侦听器。 参见https://rmoff.net/2018/08/02/kafka-listeners-explained/