Apache Beam Wait.on JdbcIO.write with unbounded PCollection issue

Question

I am trying use the below scenario with unbounded pCollection datasource(PubSub). https://issues.apache.org/jira/browse/BEAM-6732

I am able to write to DB1. DB2 is having a Wait.on DB1 (PCollection.withResults). But unfortunately DB2 is not getting updated.

When I change the source to a bounded dummy PCollection, it works. Any input is appreciated.

Answer 1

Do your writing to DB1 transforms have proper windows and triggers?

Answer 2

As I mentioned on Jira, do you have any Windows using in your unbounded pipeline? The writing to another database starts only after the respective window has been fully written to the first database. Otherwise, it will wait forever.