如何每5分钟使用spark分析一次pv，uv，ip

Question

How to analyze uv, pv, ip a day every 5 mins, and stored Mysql. Data is from Kafka in the following format:

Message sent: {"cookie":"a95f22eabc4fd4b580c011a3161a9d9d","ip":"125.119.144.252","event_time":"2017-08-07 10:50:16"}
Message sent: {"cookie":"6b67c8c700427dee7552f81f3228c927","ip":"202.109.201.181","event_time":"2017-08-07 10:50:26"}

It's just like 00:00-00:05 00:05--00:10 and so on , I used:

val write=new JDBCSink()
       val query=counts.writeStream.foreach(write).outputMode("complete")
          .trigger(ProcessingTime("5 minutes"))    
          .start()

but when I commit it on 00:01 or it's breakdown, how can I sure it not will analyse like 00:01-00:06 and so.

Answer 1

Using window function：

query = counts.groupBy(window('event_time', '5 second')).agg()
query.writeStream.start()