在 Python function 中执行参数化 BigQuery SQL

Question

我正在尝试通过 Python function 传递一些参数。在 function 内部，我正在尝试执行 BigQuery SQL 并更新现有表（创建和替换表）。 我不断得到

BadRequest: 400 1.2 - 1.118: Unrecognized token CREATE.
[Try using standard SQL (https://cloud.google.com/bigquery/docs/reference/standard-sql/enabling-standard-sql)]

(job ID: 7417d5d6-fdcd-420e-b7ac-4aaa8bb3347c)

                                              -----Query Job SQL Follows-----                                               

    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |
   1: CREATE OR REPLACE TABLE `analytics-mkt-cleanroom.MKT_DS.PXV2DWY_HS_MODEL_INTRMDT_TAB_01` AS SELECT '2021-07-01' AS DT 
    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |    .    |

错误。

这是我完整的 Jupyter Notebook 代码：

# Creating and initializing a random table:

    %%bigquery
    CREATE OR REPLACE TABLE `analytics-mkt-cleanroom.MKT_DS.Home_Services_PXV2DWY_HS_MODEL_INTRMDT_TABLE_01` AS 
    SELECT CURRENT_DATE AS DT

# Checking what's the current date:

    %%bigquery
    SELECT * FROM `analytics-mkt-cleanroom.MKT_DS.Home_Services_PXV2DWY_HS_MODEL_INTRMDT_TABLE_01`


# Initializing random str date variable:

    from_date = f"'2021-07-01'"
    to_date = f"'2022-06-30'"


# Creating a Python function to update the existing table using a parameter:

    from google.cloud import bigquery
    def my_func(from_date):
        client = bigquery.Client(project='analytics-mkt-cleanroom')
        job_config = bigquery.QueryJobConfig()
        job_config.use_legacy_sql = True   
    
        destination_table_id = f'`analytics-mkt-cleanroom.MKT_DS.PXV2DWY_HS_MODEL_INTRMDT_TAB_01`'
        
        sql = """ CREATE OR REPLACE TABLE """ + destination_table_id + """ AS SELECT {0} AS DT """.format(from_date)
        query = client.query(sql, job_config=job_config)
        query.result()
        return

# Checking what's the SQL that is getting generated inside:

    destination_table_id = f'`analytics-mkt-cleanroom.MKT_DS.PXV2DWY_HS_MODEL_INTRMDT_TAB_01`'
    sql = """ CREATE OR REPLACE TABLE """ + destination_table_id + """ AS SELECT {0} AS DT """.format(from_date)
    sql


my_func(from_date)

这只是我必须使用 Python 和 BigQuery 创建数据管道的大型项目的一小部分。

Answer 1

删除查询作业配置参数job_config.use_legacy_sql = True ，在这种情况下你不需要使用 legacy sql。实际上不需要为客户端传递作业配置，它有他的默认值。

其他可能对您有帮助的一点是使用 F-String 以获得更好的代码易读性，如下所示： sql = f"CREATE OR REPLACE TABLE {destination_table_id} AS SELECT {from_date} AS DT"