[英]Copy Firestore Database Data to Bigquery using Cloud Shell Terminal
Does anyone know how I can manually copy/transfer data from Firestore database to Bigquery using Cloud Shell Terminal?有谁知道我如何使用 Cloud Shell 终端手动将数据从 Firestore 数据库复制/传输到 Bigquery?
I did this in the past but I'm unable to find the documentation/video that I used.我过去曾这样做过,但找不到我使用的文档/视频。 I find a lot that states that once Bigquery is connected to Firebase, it should be automatic but mine is not.
我发现很多说明一旦 Bigquery 连接到 Firebase,它应该是自动的,但我的不是。
When I ran code in the Cloud Shell Terminal to pull data from Firebase the collection was copied as a table into a Bigquery dataset.当我在 Cloud Shell 终端中运行代码以从 Firebase 中提取数据时,该集合被作为表格复制到 Bigquery 数据集中。 Two tables were created and "raw_latest" and "raw_changelog" were created.
创建了两个表,并创建了“raw_latest”和“raw_changelog”。
I'm not sure how to transfer another collection now.我现在不知道如何转移另一个收藏。 I specifically need to transfer data from a subcollection in the Firestore database.
我特别需要从 Firestore 数据库中的子集合传输数据。
You can now export data from Cloud Firestore to BigQuery with a Firebase Extension.您现在可以使用 Firebase 扩展将数据从 Cloud Firestore 导出到 BigQuery。 To import all the previous data you will need first to install the extension because all the writes while doing the export first without installing the extension will be lost.
要导入所有以前的数据,您首先需要安装扩展程序,因为在不安装扩展程序的情况下先进行导出时的所有写入都将丢失。
See: https://firebase.google.com/products/extensions/firestore-bigquery-export请参阅: https://firebase.google.com/products/extensions/firestore-bigquery-export
Firestore allows import / export data to BigQuery using a GCS bucket. Firestore 允许使用 GCS 存储桶将数据导入/导出到 BigQuery。 The data is exported to a Cloud Storage bucket and from there it can be imported into Big Query.
数据会导出到 Cloud Storage 存储桶,然后可以从那里导入 Big Query。
The gcloud commands for the same are:相同的 gcloud 命令是:
export data:导出数据:
gcloud beta firestore export --collection-ids=users gs://my bucket/users
load backup into bq:将备份加载到 bq:
bq load --source_format=DATASTORE_BACKUP mydataset.users gs://gs://mybucket/users/all_namespaces/kind_users/all_namespaces_kind_users.export_metadata
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.