未使用 firestore 对大查询表进行分区 将 Collections 导出到 BigQuery 扩展

Question

I am using Export Collections to BigQuery extension with firestore to export data to the bigquery.I can export data to the big query but the table is not getting partitioned even I have enabled the table partitioned with extension while configuring extension at the creation time.

Answer 1

I had the same issue today when exporting data from Firestore to BigQuery and doing a historical load with npx firebaseextensions/fs-bq-import-collection command.

These are my findings. If you configure the extension to have partitioning based on the ingestion time, you have to wait for a first new record that will trigger that streaming extension and create a partitioned table.

If you just create an extension and immediately execute a historical load, the table created won't be partitioned. I tested this 3 times today.

A question for you -- did you execute a historical load or imported only new records?