[英]Deserializing Java org.apache.kafka.common.serialization serialized objects with Python
I have a Java Kafka producer that uses org.apache.kafka.common.serialization.LongSerializer
as the key serializer and I'm trying to consume messages from the topic using a Python Kafka consumer.我有一个 Java Kafka 生产者,它使用org.apache.kafka.common.serialization.LongSerializer
作为关键序列化程序,我正在尝试使用 Python Kafka 消费者来消费来自主题的消息。
I thought that since LongSerializer
is part of org.apache.kafka
, an equivalent serializer and deserializer would be available in all official Kafka clients for other languages, to promote interoperability.我认为由于LongSerializer
是org.apache.kafka
一部分,因此所有其他语言的官方 Kafka 客户端都可以使用等效的序列化器和反序列化器,以促进互操作性。 However, I couldn't find it.但是,我找不到它。
So, are people supposed to use org.apache.kafka.common.serialization
only for projects which are pure JVM, or is there some other way to deserialize these objects using Python?那么,人们是否应该仅将org.apache.kafka.common.serialization
用于纯 JVM 的项目,还是有其他方法可以使用 Python 反序列化这些对象?
I feel like I'm missing something because I find it hard to believe Kafka provides serializers and deserializers out of the box which do not promote communication between processes written in different languages...我觉得我错过了一些东西,因为我发现很难相信 Kafka 提供了开箱即用的序列化器和反序列化器,它们不会促进用不同语言编写的进程之间的通信......
If anyone still needs an answer a year after it was asked, you can just re-implement Java's LongSerializer in Python:如果有人在被问到一年后仍然需要答案,您可以在 Python 中重新实现 Java 的 LongSerializer:
def long_deserializer(data: bytes):
if not data:
return None
if len(data) != 8:
raise Exception(f"Size of data received by long_deserializer is not 8. Received {len(data)}")
# 0xF...FFFFFFFF is always -1
if data == b'\xff\xff\xff\xff\xff\xff\xff\xff':
return -1
value = 0
for b in data:
value <<= 8
value |= b & 0xFF
return value
The example usage would be as follows:示例用法如下:
b'\\x00\\x00\\x00\\x00\\x00\\x00\\x04\\xd2
x04\\xd2
0x04=100, d=1010 , 2=0010
10010100010
10010100010 = 1234
Also, if you want a more "pythonic" way, you can use builtin functions:此外,如果您想要更“pythonic”的方式,您可以使用内置函数:
int('0x' + data.hex().lstrip('0'), 0)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.