將來自雲存儲的外部臨時表與預先存在的大查詢表相結合 - append 來自 python

Question

我在 bigquery 中有一個永久表，我想要 append 與來自谷歌雲存儲中的 csv 的數據。 我首先將 csv 文件讀入一個大查詢臨時表：

table_id = "incremental_custs"
external_config = bigquery.ExternalConfig("CSV")
external_config.source_uris = [
    "gs://location/to/csv/customers_5083983446185_test.csv"
]
external_config.schema=schema
external_config.options.skip_leading_rows = 1
job_config = bigquery.QueryJobConfig(table_definitions={table_id: external_config})
sql_test = "SELECT * FROM `{table_id}`;".format(table_id=table_id)
query_job = bq_client.query(sql_test,job_config=job_config)
customer_updates = query_job.result()
print(customer_updates.total_rows)

直到這里一切正常，我從 tmp 表中檢索記錄。 當我嘗試將其與永久表結合時出現問題：

sql = """
create table `{project_id}.{dataset}.{table_new}` as (
      select customer_id, email, accepts_marketing, first_name, last_name,phone,updated_at,orders_count,state,
              total_spent,last_order_name,tags,ll_email,points_approved,points_spent,guest,enrolled_at,ll_updated_at,referral_id,
              referred_by,referral_url,loyalty_tier_membership,insights_segment,rewards_claimed
              from (
                select * from `{project_id}.{dataset}.{old_table}`
                union all 
                select * from `{table_id}`
                ORDER BY customer_id, orders_count  DESC
                ))
                order by orders_count desc 
""".format(project_id=project_id, dataset=dataset_id, table_new=table_new, old_table=old_table, table_id=table_id)
query_job = bq_client.query(sql)
query_result = query_job.result()

我收到以下錯誤：

BadRequest: 400 Table name "incremental_custs" missing dataset while no default dataset is set in the request.

我在這里錯過了什么嗎？ 謝謝 ！

Answer 1

Arf，你忘記了外部配置！ 您沒有在第二個腳本中傳遞它

query_job = bq_client.query(sql)

像第一個一樣簡單地更新它

query_job = bq_client.query(sql_test,job_config=job_config)

新鮮的外觀總是更容易！