如何使用扩展将所有数据从 firestore 获取到 bigquery

Question

I am new to firebase and bigquery (and all gcp services) and am trying to use the extension inside of firebase that connects my firestore db to BigQuery. I seem to have set it up correctly, but it only shows data from the time I installed the extension going forward (no older data). I would like to see all old data and any new data in my firestore DB. How can this be done?

Answer 1

From the documentation of the Stream Collections to BigQuery extension:

If you create, update, delete, or import a document in the specified collection, this extension sends that update to BigQuery. You can then run queries on this mirrored dataset.

So (as Doug commented too) it does not export the existing contents of the collection, but rather only streams the changes from the moment you install the extension.

There's a section on backfilling your BigQuery dataset that says:

This extension only sends the content of documents that have been changed -- it does not export your full dataset of existing documents into BigQuery. So, to backfill your BigQuery dataset with all the documents in your collection, you can run the import script provided by this extension.

Important: Run the import script over the entire collection after installing this extension, otherwise all writes to your database during the import might be lost.