[英]Send a json payload to Apache Kafka topic in Spring
I need to send (Post) Json payload to Apache Kafka topic but I am receiving the following error :- "message": "Can't convert value of class com.xyz.User to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer"我需要将(发布)Json 负载发送到 Apache Kafka 主题,但我收到以下错误:- “消息”:“无法将类 com.xyz.User 的值转换为类 org.apache.kafka.common.serialization .StringSerializer 在 value.serializer 中指定"
Also Spring shows class cast exception :- java.lang.ClassCastException: com.xyz.User cannot be cast to java.lang.String Spring 还显示类转换异常:- java.lang.ClassCastException: com.xyz.User cannot be cast to java.lang.String
Following is my modal,kafka config and controller以下是我的模态、kafka 配置和控制器
public class User {
private String firstname;
private String email;
public User() {}
public User(String firstname, String email) {
super();
this.firstname = firstname;
this.email = email;
}
public String getFirstname() {
return firstname;
}
public void setFirstname(String firstname) {
this.firstname = firstname;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
@Override
public String toString() {
return "UserModel [firstname=" + firstname + ", email=" + email + "]";
}
}
Kafka Configuration卡夫卡配置
package config;
import java.util.HashMap;
import java.util.Map;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringSerializer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;
import org.springframework.kafka.support.serializer.JsonSerializer;
import com.xyz.User;
@Configuration
public class KafkaConfiguration {
@Bean
public ProducerFactory<String, User> producerFactory(){
Map<String, Object> config = new HashMap<>();
config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9092");
config.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
config.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
return new DefaultKafkaProducerFactory<>(config);
}
@Bean
public KafkaTemplate<String, User> kafkaTemplate() {
return new KafkaTemplate<>(producerFactory());
}
}
Controller控制器
@RestController
@RequestMapping("/kafka")
public class UserController {
@Autowired
private KafkaTemplate<String, User> kafkaTemplate;
private static String TOPIC = "kafka-producer";
@PostMapping("/publish")
public void getUserId(@RequestBody User user) {
kafkaTemplate.send(TOPIC, user);
Json Payload send from postman从邮递员发送的 Json Payload
{
"firstname" : "xyz",
"email" : "xyz@gmail.com.com"
}
Able to reproduce your error at my local setup.能够在我的本地设置中重现您的错误。
Caused by: org.apache.kafka.common.errors.SerializationException: Can't convert value of class com.example.demo.User to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer
Caused by: java.lang.ClassCastException: com.example.demo.User cannot be cast to java.lang.String
If we look at log then it says value.serializer
value still referring to default StringSerializer
rather than expected is JsonSerializer
which in turn says your producer configuration are not getting into effect.如果我们查看日志,那么它说
value.serializer
值仍然指的是默认的StringSerializer
而不是预期的JsonSerializer
,这反过来说您的生产者配置没有生效。 In short your KafkaCOnfiguration
class is not being referred.简而言之,您的
KafkaCOnfiguration
类没有被引用。
Your KafkaConfiguration
class is in some config
package and User
class in some com.xyz package.您的
KafkaConfiguration
类位于某个 com.xyz 包中的某个config
包和User
类中。 So solution would be to make sure that it gets picks up your configuration.因此,解决方案是确保它获取您的配置。 Most probably that package may not getting scanned for configuration/beans definition.
很可能该包可能不会被扫描以获取配置/bean 定义。 If you move KafkaConfiguration to root package of your application then your original code should work fine.
如果您将 KafkaConfiguration 移动到应用程序的根包,那么您的原始代码应该可以正常工作。
If you say that your KafkaTemplate object getting injected then it's actually not.如果你说你的 KafkaTemplate 对象被注入,那么它实际上不是。 The one which is getting injected is defined by Spring kafka Autoconfiguration.
被注入的那个是由 Spring kafka Autoconfiguration 定义的。
Convert your User object to the String in producer before sending to KafkaTemplate. 在发送到KafkaTemplate之前,将您的User对象转换为生产者中的String。 And convert String to User object at the consumer's end.
并在使用者端将String转换为User对象。
Producer: 制片人:
@PostMapping("/publish")
public void getUserId(@RequestBody User user) {
kafkaTemplate.send(TOPIC, new ObjectMapper().writeValueAsString(user));
}
Consumer: 消费者:
User user = new ObjectMapper().readValue(kafkaMessage, User.class);
Service服务
@Service
public class KafkaSenderService {
@Autowired
private KafkaTemplate<String, JsonNode> kafkaTemplate;
public void send(String kafkaTopic, JsonNode message) {
kafkaTemplate.send(kafkaTopic, message);
}
}
Controller控制器
for (JsonNode data: datas) {
kafkaSender.send(topicName,data);
}
application.properties应用程序属性
spring.kafka.producer.value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
Following worked for me.以下为我工作。
public class UserController {
@Autowired
private KafkaTemplate<String, String> kafkaTemplate;
private static String TOPIC = "kafka-producer";
@PostMapping("/publish")
public void getUserId(@RequestBody User user) {
ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
String json = null;
try {
json = ow.writeValueAsString(user);
} catch (JsonProcessingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
kafkaTemplate.send(TOPIC, json);
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.