kafka 客戶端在集群重啟時獲得 TOPIC_AUTHORIZATION_FAILED

Question

我正在使用啟用了 SSL 的 kafka 集群來連接到消費者並發布消息。 下面是技術棧。

spring-kafka : 2.6.6
spring-boot  :  2.4.3

卡夫卡屬性

kafka:
 bootstrap-servers: ${BOOTSTRAP-SERVERS-HOST}
 subscription-topic: TEST
 properties:
   security.protocol: SSL
   ssl.truststore.location: ${SUBSCRIPTION_TRUSTSTORE_PATH}
   ssl.truststore.password: ${SUBSCRIPTION_TRUSTSTORE_PWD}
   ssl.keystore.location: ${SUBSCRIPTION_KEYSTORE_PATH}
   ssl.keystore.password: ${SUBSCRIPTION_KEYSTORE_PWD}

問題： Kafka 客戶端應用程序已連接到 kafka 集群消費者並按預期發布消息。
現在我們停止以下錯誤記錄的 kafka 代理/集群。

could not be established. Broker may not be available.

這很好，並且可以預期，因為代理/集群已關閉。

現在我們啟動代理/集群，下面的錯誤開始出現，kafka 消費者停止消費來自主題的消息，但是 kafka 發布者能夠向主題發送消息。 [應用程序重啟解決了這個問題]

非常感謝嘗試了解根本原因的任何幫助。

2022-01-13 13:34:52.078 [TEST.CONSUMER-GROUP-0-C-1] ERROR--SUBSCRIPTION -  -org.apache.kafka.clients.Metadata.checkUnauthorizedTopics - [Consumer clientId=consumer-TEST.CONSUMER-GROUP-1, groupId=TEST.CONSUMER-GROUP] Topic authorization failed for topics [TEST]
2022-01-13 13:34:52.078 [TEST.CONSUMER-GROUP-0-C-1] ERROR- -SUBSCRIPTION -  -org.springframework.core.log.LogAccessor.error - Authorization Exception and no authorizationExceptionRetryInterval set
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [TEST]

2022-01-13 13:34:52.081 [TEST.CONSUMER-GROUP-0-C-1] ERROR- IRVS-SUBSCRIPTION -  -org.springframework.core.log.LogAccessor.error - Fatal consumer exception; stopping container
2022-01-13 13:34:52.083 [TEST.CONSUMER-GROUP-0-C-1] INFO - IRVS-SUBSCRIPTION -  -org.springframework.scheduling.concurrent.ExecutorConfigurationSupport.shutdown - Shutting down ExecutorService

Answer 1

添加后上述問題已解決

AuthorizationException RetryInterval

下面是一個例子來說明這一點

 @Bean
   ConcurrentKafkaListenerContainerFactory<Object, Object> kafkaListenerContainerFactory(
           ConcurrentKafkaListenerContainerFactoryConfigurer configurer,
           ConsumerFactory<Object, Object> kafkaConsumerFactory) {
       ConcurrentKafkaListenerContainerFactory<Object, Object> factory =
            new ConcurrentKafkaListenerContainerFactory<>();
       factory.setConcurrency(2);
       configurer.configure(factory, kafkaConsumerFactory);
       // other setters like error handler , retry handler
       // setting Authorization Exception Retry Interval 
       factory.getContainerProperties()
               .setAuthorizationExceptionRetryInterval(Duration.ofSeconds(5l));

       return factory;
   }