GCP Dataflow - How to read the data from Google BigQuery and load into Google Spanner using Dataflow

Question

The problem statement is to Load the Google Spanner data from Google BigQuery, The size of BigQuery table is huge and we need to come up with the effective solution

I tried exploring various options but not getting the solution

Answer 1

Google provides many templates to move data from one storage system to another. You can find them at https://cloud.google.com/dataflow/docs/guides/templates/provided-templates

None of these are "BigQuery to Cloud Spanner" but they are very similar. The templates are open source, so you can use them as examples. You can find the source code for the templates at https://github.com/GoogleCloudPlatform/DataflowTemplates