[英]The meaning of the content of the __consumer_offsets topic
我正在使用 kafka-dump-log 工具查看 __consumer_offsets 主题的内容。 有人可以解释该内容的确切含义(offset、createtime、isvalid、keysize、valuesize、compresscodec、producerId、producerEpoch、isTransactional、headerKeys、key、offset、payload)吗?
[root@localhost kafka_2.11-2.0.0]# bin/kafka-dump-log.sh --files data/kafka/__consumer_offsets-32/00000000000000036360.log --value-decoder-class "kafka.serializer.StringDecoder" --offsets-decoder
Dumping data/kafka/__consumer_offsets-32/00000000000000036360.log
Starting offset: 36360
offset: 36360 position: 0 CreateTime: 1623678492136 isvalid: true keysize: 19 valuesize: 28 magic: 2 compresscodec: NONE producerId: -1 producerEpoch: -1 sequence: 0 isTransactional: false headerKeys: [] key: offset::b0147:test:48 payload: 35
offset: 36361 position: 0 CreateTime: 1623678492136 isvalid: true keysize: 19 valuesize: 28 magic: 2 compresscodec: NONE producerId: -1 producerEpoch: -1 sequence: 1 isTransactional: false headerKeys: [] key: offset::b0147:test:64 payload: 34
offset: 36362 position: 0 CreateTime: 1623678492136 isvalid: true keysize: 19 valuesize: 28 magic: 2 compresscodec: NONE producerId: -1 producerEpoch: -1 sequence: 2 isTransactional: false headerKeys: [] key: offset::b0147:test:14 payload: 34
...
offset: 36460 position: 5497 CreateTime: 1623678503139 isvalid: true keysize: 9 valuesize: 24 magic: 2 compresscodec: NONE producerId: -1 producerEpoch: -1 sequence: -1 isTransactional: false headerKeys: [] key: {"metadata":"b0147"} payload: {"protocolType":"consumer","protocol":null,"generationId":2,"assignment":"{}"}
offset: 36461 position: 5598 CreateTime: 1623678554656 isvalid: true keysize: 19 valuesize: -1 magic: 2 compresscodec: NONE producerId: -1 producerEpoch: -1 sequence: -1 isTransactional: false headerKeys: []
...
您看到的字段并非特定于__consumer_offsets
,它们只是日志中每条消息/批次的元数据,您将获得所有主题的元数据,请参阅https://kafka.apache.org/documentation/#recordbatch
如果您想探索__consumer_offsets
的内容,您需要知道它包含 2 种类型的记录:
要显示 OffsetMetadata,您可以使用:
./bin/kafka-console-consumer.sh \
--formatter "kafka.coordinator.group.GroupMetadataManager\$OffsetsMessageFormatter" \
--bootstrap-server localhost:9092 --topic __consumer_offsets
要显示 GroupMetadata,您可以使用:
./bin/kafka-console-consumer.sh \
--formatter "kafka.coordinator.group.GroupMetadataManager\$GroupMetadataMessageFormatter" \
--bootstrap-server localhost:9092 --topic __consumer_offsets
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.