[英]Is there a code sample for multiple producers in spring kafka?
I have an application that may need multiple producers.我有一个可能需要多个生产者的应用程序。 All code samples I see seem to support a single producer, reading config from app during app startup.
我看到的所有代码示例似乎都支持单个生产者,在应用程序启动期间从应用程序读取配置。 If there are multiple producers and we want to pass in different producer config, is there out of the box support in Spring?
如果有多个生产者并且我们想传入不同的生产者配置,那么 Spring 是否有开箱即用的支持? Or should I just go without spring in that case?
或者在这种情况下我应该没有弹簧吗?
You can create several Producer
instances ( KafkaTemplate
) via the same ProducerFactory
. 您可以通过同一
ProducerFactory
创建多个Producer
实例( KafkaTemplate
)。
If you need different Kafka configurations, you'll need different ProducerFactory
instances. 如果您需要不同的Kafka配置,则需要不同的
ProducerFactory
实例。
you will have to create two different ProducerFactory
below is example 您将必须创建两个不同的
ProducerFactory
下面是示例
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringSerializer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;
import java.util.HashMap;
@Configuration
public class KafkaProducerConfig {
@Bean
public ProducerFactory<String, String> confluentProducerFactory() {
HashMap<String, Object> configProps = new HashMap<String, Object>();
configProps.put(
ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
"localhost:9092");
configProps.put(
ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
StringSerializer.class);
configProps.put(
ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
StringSerializer.class);
return new DefaultKafkaProducerFactory<>(configProps);
}
@Bean
public ProducerFactory<String, String> cloudraProducerFactory() {
HashMap<String, Object> configProps = new HashMap<String, Object>();
configProps.put(
ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
"localhost:9094");
configProps.put(
ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
StringSerializer.class);
configProps.put(
ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
StringSerializer.class);
return new DefaultKafkaProducerFactory<>(configProps);
}
@Bean(name = "confluent")
public KafkaTemplate<String, String> confluentKafkaTemplate() {
return new KafkaTemplate<>(confluentProducerFactory());
}
@Bean(name = "cloudera")
public KafkaTemplate<String, String> clouderaKafkaTemplate() {
return new KafkaTemplate<>(cloudraProducerFactory());
}
}
public class ProducerExample {
@Autowired
@Qualifier("cloudera")
private KafkaTemplate clouderaKafkaTemplate;
@Autowired
@Qualifier("confluent")
private KafkaTemplate confluentKafkaTemplate;
public void send() {
confluentKafkaTemplate.send("TestConfluent", "hey there..confluent");
clouderaKafkaTemplate.send("TestCloudEra","hey there.. cloudera");
}
}
If you still want to keep your configuration in application.yaml
as usual, and keep Java configuration as minimum as possible, you can extend KafkaProperties.Producer
.如果您仍然想像往常一样在
application.yaml
保留您的配置,并尽可能减少 Java 配置,您可以扩展KafkaProperties.Producer
。
@Configuration
@ConfigurationProperties(prefix = "spring.kafka.producer-1")
@RequiredArgsConstructor
class FirstProducer extends KafkaProperties.Producer {
private final KafkaProperties common;
@Qualifier("producer-1")
@Bean
public ProducerFactory<?, ?> producerFactory() {
final var conf = new HashMap<>(
this.common.buildProducerProperties()
);
conf.putAll(this.buildProperties());
return new DefaultKafkaProducerFactory<>(conf);
}
@Qualifier("producer-1")
@Bean
public KafkaTemplate<?, ?> kafkaTemplate() {
return new KafkaTemplate<>(this.producerFactory());
}
}
@Configuration
@ConfigurationProperties(prefix = "spring.kafka.producer-2")
@RequiredArgsConstructor
class SecondProducer extends KafkaProperties.Producer {
private final KafkaProperties common;
@Qualifier("producer-2")
@Bean
public ProducerFactory<?, ?> producerFactory() {
final var conf = new HashMap<>(
this.common.buildProducerProperties()
);
conf.putAll(this.buildProperties());
return new DefaultKafkaProducerFactory<>(conf);
}
@Qualifier("producer-2")
@Bean
public KafkaTemplate<?, ?> kafkaTemplate() {
return new KafkaTemplate<>(this.producerFactory());
}
}
Starting with version 2.5, you can use a RoutingKafkaTemplate to select the producer at runtime, based on the destination topic name.从版本 2.5 开始,您可以使用 RoutingKafkaTemplate 在运行时根据目标主题名称选择生产者。 https://docs.spring.io/spring-kafka/reference/html/#routing-template
https://docs.spring.io/spring-kafka/reference/html/#routing-template
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.