[英]How to send Custom Object to Kafka Topic with Producer
I would like to send my Account class with Producer to my Kafka Topic, then I will aggregate with Kafka Stream.我想将我的 Account 类和 Producer 发送到我的 Kafka Topic,然后我将与 Kafka Stream 聚合。 However, I can not send Object I am getting error :
但是,我无法发送对象我收到错误:
Caused by: org.apache.kafka.common.KafkaException: bank.Account is not an instance of org.apache.kafka.common.serialization.Serializer引起:org.apache.kafka.common.KafkaException:bank.Account 不是 org.apache.kafka.common.serialization.Serializer 的实例
My Producer class:我的制作人课程:
public static void main(String[] args) {
DataAccess dataAccess = new DataAccess();
List<Account> accountList = dataAccess.read();
final Logger logger = LoggerFactory.getLogger(Producer.class);
Properties properties = new Properties();
properties.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"127.0.0.1:9092");
properties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,LongSerializer.class.getName());
properties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,Account.class.getName());
KafkaProducer<Long,Account> producer = new KafkaProducer<Long, Account>(properties);
for (Account account : accountList) {
ProducerRecord<Long,Account> record = new ProducerRecord<Long, Account>("bank_account",account.getFromId(),account);
producer.send(record, new Callback() {
public void onCompletion(RecordMetadata recordMetadata, Exception e) {
if (e == null) {
logger.info("Record sent successfully. \n "+ "Topic : "+recordMetadata.topic() +"\n"+
"Partition : " + recordMetadata.partition() + "\n"+
"Offset : " +recordMetadata.offset() +"\n"+
"Timestamp: " +recordMetadata.timestamp() +"\n");
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
}
else{
logger.info("Error sending producer");
}
}
});
}
producer.flush();
producer.close();
}
It gives error in this line :它在这一行给出错误:
KafkaProducer<Long,Account> producer = new KafkaProducer<Long, Account>(properties);
My Account class:我的帐户类:
public class Account {
private long fromId;
private long amount;
private long toId;
private ZonedDateTime time;
}
So my question is, how can we send custom object to kafka topics?所以我的问题是,我们如何将自定义对象发送到 kafka 主题? After that I want to consume that message of course.
在那之后,我当然想使用该消息。
This line这条线
properties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,Account.class.getName());
You must implement your own Serializer
class .您必须实现自己的
Serializer
类。 It cannot be a plain class.它不能是一个普通的类。
Some people use JSON to serialize, others use Avro or Protobuf.有些人使用 JSON 进行序列化,有些人使用 Avro 或 Protobuf。 However you get the data into a
byte[]
is just an implementation detail.但是,您将数据放入
byte[]
只是一个实现细节。
//1
prop.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
prop.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, Employee.class.getName());
//2
KafkaProducer<String, Employee> producer = new KafkaProducer(prop);
Employee emp = new Employee(1, "Arun");
ProducerRecord prodRecord = new ProducerRecord("aryan_topic", emp);
//3
import org.apache.kafka.common.header.Headers;
import org.apache.kafka.common.serialization.Serializer;
import java.io.Serializable;
import java.util.Map;
//Developed by Arun Singh
public class Employee implements Serializable, Serializer {
Integer empId;
String empName;
Address add;
public Employee() {
}
public Employee(Integer empId, String empName, Address add) {
this.empId = empId;
this.empName = empName;
this.add = add;
}
public Integer getEmpId() {
return empId;
}
public String getEmpName() {
return empName;
}
public Address getAdd() {
return add;
}
public void setEmpId(Integer empId) {
this.empId = empId;
}
public void setEmpName(String empName) {
this.empName = empName;
}
public void setAdd(Address add) {
this.add = add;
}
public void configure(Map configs, boolean isKey) {
}
public byte[] serialize(String s, Object o) {
return new byte[0];
}
public byte[] serialize(String topic, Headers headers, Object data) {
return new byte[0];
}
public void close() {
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.