在 Python function 中執行參數化 BigQuery SQL

Question

我正在嘗試通過 Python function 傳遞一些參數。在 function 內部，我正在嘗試執行 BigQuery SQL 並更新現有表（創建和替換表）。 我不斷得到

BadRequest: 400 1.2 - 1.118: Unrecognized token CREATE.
[Try using standard SQL (https://cloud.google.com/bigquery/docs/reference/standard-sql/enabling-standard-sql)]

(job ID: 7417d5d6-fdcd-420e-b7ac-4aaa8bb3347c)

                                              -----Query Job SQL Follows-----                                               

    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |
   1: CREATE OR REPLACE TABLE `analytics-mkt-cleanroom.MKT_DS.PXV2DWY_HS_MODEL_INTRMDT_TAB_01` AS SELECT '2021-07-01' AS DT 
    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |

錯誤。

這是我完整的 Jupyter Notebook 代碼：

# Creating and initializing a random table:

    %%bigquery
    CREATE OR REPLACE TABLE `analytics-mkt-cleanroom.MKT_DS.Home_Services_PXV2DWY_HS_MODEL_INTRMDT_TABLE_01` AS 
    SELECT CURRENT_DATE AS DT

# Checking what's the current date:

    %%bigquery
    SELECT * FROM `analytics-mkt-cleanroom.MKT_DS.Home_Services_PXV2DWY_HS_MODEL_INTRMDT_TABLE_01`


# Initializing random str date variable:

    from_date = f"'2021-07-01'"
    to_date = f"'2022-06-30'"


# Creating a Python function to update the existing table using a parameter:

    from google.cloud import bigquery
    def my_func(from_date):
        client = bigquery.Client(project='analytics-mkt-cleanroom')
        job_config = bigquery.QueryJobConfig()
        job_config.use_legacy_sql = True   
    
        destination_table_id = f'`analytics-mkt-cleanroom.MKT_DS.PXV2DWY_HS_MODEL_INTRMDT_TAB_01`'
        
        sql = """ CREATE OR REPLACE TABLE """ + destination_table_id + """ AS SELECT {0} AS DT """.format(from_date)
        query = client.query(sql, job_config=job_config)
        query.result()
        return

# Checking what's the SQL that is getting generated inside:

    destination_table_id = f'`analytics-mkt-cleanroom.MKT_DS.PXV2DWY_HS_MODEL_INTRMDT_TAB_01`'
    sql = """ CREATE OR REPLACE TABLE """ + destination_table_id + """ AS SELECT {0} AS DT """.format(from_date)
    sql


my_func(from_date)

這只是我必須使用 Python 和 BigQuery 創建數據管道的大型項目的一小部分。

Answer 1

刪除查詢作業配置參數job_config.use_legacy_sql = True ，在這種情況下你不需要使用 legacy sql。實際上不需要為客戶端傳遞作業配置，它有他的默認值。

其他可能對您有幫助的一點是使用 F-String 以獲得更好的代碼易讀性，如下所示： sql = f"CREATE OR REPLACE TABLE {destination_table_id} AS SELECT {from_date} AS DT"