为什么在 apache-beam 中出现错误：“TypeError：使用 SessionWindow 时无法将 GlobalWindow 转换为 _IntervalWindowBase？

Question

When I use session window with 1h gap and after processing million of messages I get error in logs, probably just for some rows:

TypeError: Cannot convert GlobalWindow to apache_beam.utils.windowed_value._IntervalWindowBase

Code:

grouped_tis = tracking_informations | beam.WindowInto(window.Sessions(session_window_gap)) | beam.GroupByKey() | beam.ParDo(MergeTI()) | "TI model -> json" >> beam.Map(jsons.dump)

Full stack: https://pastebin.com/pqA5pMay

Answer 1

这可能是因为某些代码（例如MergeTI ）正在返回GlobalWindow元素，而PCollection具有不同的窗口集： beam.WindowInto(window.Sessions(session_window_gap)) 。

Answer 2

If anyone is experiencing same problem, I solve this problem insert beam.WindowInto(beam.window.GlobalWindows()) between beam.WindowInto(NONGLOBALWINDOW) | beam.GroupByKey() beam.WindowInto(NONGLOBALWINDOW) | beam.GroupByKey() and other Ptransform that causes problem.