在 Bigquery 中使用 Job Config 创建范围分区表

Question

尝试将 csv 文件读入 dataframe 并使用该 dataframe 加载到具有范围分区的 Bigquery 表中。 但是为 Long: 获得 400 POST Invalid value 不是正确的类型错误。

重现步骤：使用 google-cloud-bigquery v1.24.0

Test.csv

Name, Age, DOB
"rona", 10, 01-01-2010
"king", 20, 05-01-2000

这是要复制的代码

import pandas as pd
from google.cloud import bigquery

def Range_Partitioning(field, dict_range):
    cRangePartition = bigquery.RangePartitioning(range_=bigquery.PartitionRange(start=dict_range.get("Start"), interval=dict_range.get("Interval"), end=dict_range.get("End")),
                field=field)
    return cRangePartition

df = pd.read_csv("Test.txt", dtype={ "Name": "str", "Age": "int64", "DOB": "str"}, parse_dates=["DOB"])
BQClient = bigquery.Client()
Dataset = "Test"
TableName = "Load_Range_Test"
schema = [
    {
        "name": "Name",
        "type": "STRING",
        "mode": "REQUIRED"
    },
    {
        "name": "Age",
        "type": "INTEGER",
        "mode": "REQUIRED"
    },
    {
        "name": "DOB",
        "type": "DATE",
        "mode": "REQUIRED"
    }
]
TableRef = sProjectId + "." + Dataset + "." + TableName
RangePartition = Range_Partitioning("Age", {"start":0,  "interval":1, "end":100})
WriteOption = "WRITE_TRUNCATE"
JobConfig = bigquery.LoadJobConfig(
                    schema=schema,
                    write_disposition=WriteOption,
                    range_partitioning=RangePartition)
Job = BQClient.load_table_from_dataframe(df, TableRef, job_config=JobConfig)
Job.result()

错误：400 POST Invalid value for Long：不是正确的类型

当我不进行范围分区时工作，只有使用范围分区我才会收到此错误。

Answer 1

我已经完成代码正常工作，省略Range_Partitioning(field, dict_range) function 用法，明确指定bigquery.RangePartitioning参数：

RangePartition = bigquery.RangePartitioning(
field="Age",
range_=bigquery.PartitionRange(start=0, end=100, interval=1)
)
WriteOption = "WRITE_TRUNCATE"
JobConfig = bigquery.LoadJobConfig(
                    schema=schema,
                    write_disposition=WriteOption,
                    range_partitioning=RangePartition)
Job = BQClient.load_table_from_dataframe(df, TableRef, job_config=JobConfig)
Job.result()

似乎是cRangePartition返回不一致的范围值。