[英]spring kafka - multiple consumer reading from a single topic
I am building a web application using spring boot and now I have this requirement of receiving real-time notifications.我正在使用 Spring Boot 构建一个 Web 应用程序,现在我有接收实时通知的要求。 I am planning to use apache kafka as a message broker for this.
我打算为此使用 apache kafka 作为消息代理。 The requirement is such that there are users with different roles and based on the role, they should receive notifications of what other users are doing.
要求是存在具有不同角色的用户,并且基于角色,他们应该收到其他用户正在做什么的通知。
I did set up a single producer and consumer and as a consumer, I could receive the information published to a topic let's say topic1.我确实设置了一个生产者和消费者,作为消费者,我可以接收发布到主题的信息,比如 topic1。
The part where I am stuck is that I could have several users listening to the same topic and each user should get the message published to that topic.我被卡住的部分是我可以让多个用户收听同一主题,并且每个用户都应该将消息发布到该主题。 I understand that for this requirement, we need to set different group.id for each kafkalistener so that each consumer can get the message.
我理解对于这个需求,我们需要为每个kafkalistener设置不同的group.id,以便每个消费者都能得到消息。
But how I am going to create a kafkalistener with a different group id when a user is logged in?但是,当用户登录时,我将如何创建具有不同组 ID 的 kafkalistener? Hope someone can provide some guidance on that?
希望有人可以提供一些指导? Thank you
谢谢
Simply create a new KafkaMessageListenerContainer
each time and start/stop it as needed.每次只需创建一个新的
KafkaMessageListenerContainer
并根据需要启动/停止它。
You can use Boot's auto-configured ConcurrentKafkaListenerContainerFactory
to create containers.您可以使用 Boot 的自动配置
ConcurrentKafkaListenerContainerFactory
来创建容器。 Just set the groupId
container property to make them unique.只需设置
groupId
容器属性以使其唯一。
EDIT编辑
Here's an example:下面是一个例子:
@SpringBootApplication
public class So60150686Application {
public static void main(String[] args) {
SpringApplication.run(So60150686Application.class, args);
}
@Bean
public ApplicationRunner runner(KafkaTemplate<String, String> template) {
return args -> {
template.send("so60150686", "foo");
};
}
@Bean
public NewTopic topic() {
return TopicBuilder.name("so60150686").partitions(1).replicas(1).build();
}
}
@RestController
class Web {
private final ConcurrentKafkaListenerContainerFactory<String, String> factory;
public Web(ConcurrentKafkaListenerContainerFactory<String, String> factory) {
this.factory = factory;
}
@GetMapping(path="/foo/{group}")
public String foo(@PathVariable String group) {
ConcurrentMessageListenerContainer<String, String> container = factory.createContainer("so60150686");
container.getContainerProperties().setGroupId(group);
container.getContainerProperties().setMessageListener(new MessageListener<String, String>() {
@Override
public void onMessage(ConsumerRecord<String, String> record) {
System.out.println(record);
}
});
container.start();
return "ok";
}
}
spring.kafka.consumer.auto-offset-reset=earliest
$ http localhost:8080/foo/bar
HTTP/1.1 200
Connection: keep-alive
Content-Length: 2
Content-Type: text/plain;charset=UTF-8
Date: Mon, 10 Feb 2020 19:42:02 GMT
Keep-Alive: timeout=60
ok
2020-02-10 14:42:09.744 INFO 34096 --- [ consumer-0-C-1] osklKafkaMessageListenerContainer : bar: partitions assigned: [so60150686-0]
2020-02-10 14:42:09.744 INFO 34096 --- [consumer-0-C-1] osklKafkaMessageListenerContainer:bar:分配的分区:[so60150686-0]
ConsumerRecord(topic = so60150686, partition = 0, leaderEpoch = 0, offset = 1, CreateTime = 1581363648938, serialized key size = -1, serialized value size = 3, headers = RecordHeaders(headers = [], isReadOnly = false), key = null, value = foo)
ConsumerRecord(topic = so60150686, partition = 0, leaderEpoch = 0, offset = 1, CreateTime = 1581363648938, 序列化key size = -1, serialized value size = 3, headers = RecordHeaders(headers = [], isReadOnly = false), key = 空,值 = foo)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.