How to analyze uv, pv, ip a day every 5 mins, and stored Mysql. Data is from Kafka in the following format:
Message sent: {"cookie":"a95f22eabc4fd4b580c011a3161a9d9d","ip":"125.119.144.252","event_time":"2017-08-07 10:50:16"}
Message sent: {"cookie":"6b67c8c700427dee7552f81f3228c927","ip":"202.109.201.181","event_time":"2017-08-07 10:50:26"}
It's just like 00:00-00:05 00:05--00:10 and so on , I used:
val write=new JDBCSink()
val query=counts.writeStream.foreach(write).outputMode("complete")
.trigger(ProcessingTime("5 minutes"))
.start()
but when I commit it on 00:01 or it's breakdown, how can I sure it not will analyse like 00:01-00:06 and so.
Using window
function:
query = counts.groupBy(window('event_time', '5 second')).agg()
query.writeStream.start()
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.