使用 spring-kafka 在一天中的特定时间从一个主题消费

Question

我们需要在一天中的特定时间使用来自特定主题的数据，我们希望我们的 kafka 侦听器仅在特定主题的一天中的那个时间工作。

正在阅读 Kafka Scheduler https://jaceklaskowski.gitbooks.io/apache-kafka/kafka-KafkaScheduler.html的文章

任何其他方法或使用 Kafka 调度程序的任何帮助都值得赞赏。 我们将使用 spring kafka 1.3.10.RELEASE 版本

Answer 1

KafkaListenerEndpointRegistry并调用start()/stop()来启动/停止所有容器。

您还可以使用registry.getListenerContainer("foo").start()来启动具有特定id的容器。

在容器上将autoStartup设置为 false，以便 Spring 在启动期间不会自动启动它们。

Answer 2

如果我们使用 start() 在容器上方启动 kafka 容器，我们将低于 null 指针错误。

2019-10-28 19:22:00.011 [pool-1-thread-1] DEBUG c.a.m.r.k.config.KafkaListenerConfig -
                *********:  Zone time after kafka error listener is started 2019-10-28T19:00+05:30[Asia/Kolkata]
Listner list ConcurrentMessageListenerContainer [concurrency=12, beanName=org.springframework.kafka.KafkaListenerEndpointContainer#0, running=true]
Listner list ConcurrentMessageListenerContainer [concurrency=12, beanName=org.springframework.kafka.KafkaListenerEndpointContainer#1, running=false]
2019-10-28 19:24:08.187 [pool-1-thread-1] ERROR o.s.s.s.TaskUtils$LoggingErrorHandler -
                Unexpected error occurred in scheduled task.
java.lang.NullPointerException: null
    at com.anthem.mbr.reg.kafka.config.KafkaListenerConfig.initiateKafkaErrorListener(KafkaListenerConfig.java:110)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

下面是代码

@Autowired KafkaListenerEndpointRegistry kafkaListenerRegistry;

@Scheduled(cron = "0 22 19 * * ?")
public void initiateKafkaErrorListener() {
    log.info("Cron Task :: Execution Start Time - {}", dateTimeFormatter.format(LocalDateTime.now()));
    ZonedDateTime zoneTime = KafkaConsumerUtil.getZoneTime(zoneId);
    log.debug("{} Zone time after kafka error listener is started {}", ApplicationConstant.*****, zoneTime);
    for (MessageListenerContainer listeners : kafkaListenerRegistry.getListenerContainers()) {
        System.out.println("Listner list "+ listeners);

    }
    kafkaListenerRegistry.getListenerContainer("kafkaErrorContainerFactory").start();
}


@KafkaListener(topics = ********}", 
                containerFactory = "kafkaErrorContainerFactory")
public void receiveErrorMessages(ConsumerRecord<?, ?> consumerRecord, Acknowledgment acknowledgment) {
    try {
        ObjectMapper mapper = new ObjectMapper();
        Event event = mapper.readValue(((String)consumerRecord.value()), *****.class); }

对此有任何建议，获取容器时未解析 bean 名称

Listner list ConcurrentMessageListenerContainer [concurrency=12, beanName=org.springframework.kafka.KafkaListenerEndpointContainer#0, running=false]
Listner list ConcurrentMessageListenerContainer [concurrency=12, beanName=org.springframework.kafka.KafkaListenerEndpointContainer#1, running=true]

所以容器是 null

Container:: null