使用服务帐户导出 GCP 数据流 SQL

Question

I have a service account access to a BQ view and I want to query it and export the data to GCS or a BQ dataset in another project.

Dataflow (+create job from SQL) looks perfect for the use case but I can't work out how to set it up so the service account is used for the query. It appears Dataflow accepts using a service account but how is this done?

Thanks in advance,

Answer 1

If you want to export data to another BigQuery table, you can simply use an INSERT INTO <table> SELECT * FROM... (also known as INSERT-SELECT). If you need to export this regularly, you can use scheduled queries

if you want to export to Google Cloud Storage, you have 2 solutions

• Export a full table, and use BigQuery export API (you can call this API with Cloud Scheduler if you want to regularly export them)
• Use export query statement.