[英]How to get window start or end time from kafka topic corresponding to KSQL windowed table?
I want to get window start timestamp (here is 1530008520000) from kafka-console-consumer command.我想从 kafka-console-consumer 命令获取窗口开始时间戳(这里是 1530008520000)。
It works with KSQL:它适用于 KSQL:
ksql> select * from DEV_MONITOR_RULE_2557_104782_233_2_TABLE;
1530008581051 | 2557 : Window{start=1530008520000 end=-} | 2557 | 2 | 1530008581051
But it does not work with kafka-console-consumer?但它不适用于 kafka-console-consumer?
./bin/kafka-console-consumer --zookeeper 10.12.0.157:2181 --topic DEV_MONITOR_RULE_2557_104782_233_2_TABLE
{"HITCOUNTS":2,"TENANTID":2557,"HITTIME":1530008581051}
How to print window start time (here is 1530008520000) from kafka-console-consumer?如何从 kafka-console-consumer 打印窗口开始时间(这里是 1530008520000)?
Thanks!谢谢!
The window start time is reflected in the ROWTIME
of the KSQL message, and in the timestamp of the Kafka message.窗口开始时间被反映在
ROWTIME
的KSQL消息,并且在消息卡夫卡的时间戳。 This timestamp you can access using the standard APIs for working with Kafka messages.您可以使用用于处理 Kafka 消息的标准 API 访问此时间戳。
AFAIK the kafka-console-consumer
doesn't support showing timestamp. AFAIK
kafka-console-consumer
不支持显示时间戳。 However something like kafkacat
does.但是像
kafkacat
这样的东西。
Here's some aggregated data in KSQL, with the ROWTIME
showing the start of the window (and TIMESTAMPTOSTRING
being used to pretty-print it):这是 KSQL 中的一些聚合数据,其中
ROWTIME
显示窗口的开始(并且TIMESTAMPTOSTRING
用于漂亮地打印它):
ksql> SELECT TIMESTAMPTOSTRING(ROWTIME, 'yyyy-MM-dd HH:mm:ss'), ROWTIME, STARS, STAR_COUNT FROM RATINGS_AGG2;
2018-06-27 09:30:00 | 1530091800000 | 1 | 2
2018-06-27 09:30:00 | 1530091800000 | 4 | 6
2018-06-27 09:30:00 | 1530091800000 | 2 | 2
2018-06-27 09:30:00 | 1530091800000 | 3 | 3
Now the same topic in kafkacat
:现在
kafkacat
的相同主题:
$ kafkacat -b localhost:9092 -C -K: \
-f '\nTimestamp: %T\t\tValue (%S bytes): %s' \
-t RATINGS_AGG2
Timestamp: 1530091800000 Value (26 bytes): {"STAR_COUNT":1,"STARS":1}
Timestamp: 1530091800000 Value (26 bytes): {"STAR_COUNT":2,"STARS":1}
Timestamp: 1530091800000 Value (26 bytes): {"STAR_COUNT":3,"STARS":3}
Timestamp: 1530091800000 Value (26 bytes): {"STAR_COUNT":5,"STARS":1}
Timestamp: 1530091800000 Value (26 bytes): {"STAR_COUNT":6,"STARS":3}
Timestamp: 1530091800000 Value (26 bytes): {"STAR_COUNT":7,"STARS":1}
Timestamp: 1530091800000 Value (26 bytes): {"STAR_COUNT":7,"STARS":3}
Timestamp: 1530091800000 Value (26 bytes): {"STAR_COUNT":9,"STARS":1}
Timestamp: 1530091800000 Value (26 bytes): {"STAR_COUNT":9,"STARS":3}
您是否尝试过 kafka-console-consumer 选项print.timestamp=true
?
$ kafka-console-consumer --property print.timestamp=true ...
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.