[英]Deserializing Java objects from Kafka consumer
I have a Kafka Consumer, currently configured with: 我有一个Kafka Consumer,目前配置有:
kafkaProps.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
kafkaProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
But what I really want is to be able to use a Kryo Deserializer instead: 但我真正想要的是能够使用Kryo Deserializer代替:
public class KryoPOJODeserializer<T> implements Deserializer<T> {
private Kryo kryo = new Kryo();
@Override
public void configure(Map props, boolean isKey) {
kryo.setInstantiatorStrategy(new DefaultInstantiatorStrategy(new StdInstantiatorStrategy()));
kryo.register( Arrays.asList( "" ).getClass(), new ArraysAsListSerializer() );
}
@Override
public T deserialize(String topic, byte[] data) {
// Deserialize the serialized object.
return kryo.readObject(new Input(data), T.class);
}
@Override
public void close() {
}
}
What I can't figure out is, is it possible to reuse the same Consumer for different topics (each topic has a different type of POJO on it)? 我无法弄清楚的是,是否有可能为不同的主题重用相同的消费者(每个主题上都有不同类型的POJO)? If my consumer config is: 如果我的消费者配置是:
kafkaProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, KryoPOJODeserializer.class.getName());
Or, do I have to have a separate Consumer for every topic? 或者,我是否必须为每个主题都有一个单独的消费者?
Alternatively, do I have to remove the generics part of my Deserializer, always return an Object, and cast the Object into the relevant POJO in the client code? 或者,我是否必须删除反序列化器的泛型部分,始终返回一个Object,并将Object转换为客户端代码中的相关POJO? Something like: 就像是:
public class KryoPOJODeserializer implements Deserializer {
private Kryo kryo = new Kryo();
@Override
public void configure(Map props, boolean isKey) {
kryo.setInstantiatorStrategy(new DefaultInstantiatorStrategy(new StdInstantiatorStrategy()));
kryo.register( Arrays.asList( "" ).getClass(), new ArraysAsListSerializer() );
}
@Override
public Object deserialize(String topic, byte[] data) {
// Deserialize the serialized object.
return kryo.readClassAndObject(new Input(new ByteArrayInputStream(data)));
}
@Override
public void close() {
}
}
The latter would work but it feels a bit dirty. 后者会工作,但感觉有点脏。
Any suggestions appreciated! 任何建议赞赏!
You can use you original approach by passing Deserializer
instances directly into the Consumer: 您可以通过将Deserializer
实例直接传递给Consumer来使用原始方法:
KafkaConsumer<String, Foo> consumer = new KafkaConsumer<>(properties,
new StringDeserializer(), new KryoPOJODeserializer(Foo.class));
If you want to reuse the same incoming data type for a number of topics then you can set up a subscription to those topics using a single consumer. 如果要为多个主题重用相同的传入数据类型,则可以使用单个使用者设置对这些主题的订阅。 If you want different object types for the values, then you will need to use multiple consumers. 如果您想要值的不同对象类型,那么您将需要使用多个使用者。
Otherwise your second approach is also valid. 否则你的第二种方法也是有效的。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.