使用 Python bigquery_datatransfer 創建 Adwords BigQuery Transfer：錯誤 400 請求包含無效參數

Question

我的步驟：

1. 獲取授權碼：我使用這個網址來獲取授權碼

   https://www.gstatic.com/bigquerydatatransfer/oauthz/auth?clientId=<id from list data sources>&scope=https://www.googleapis.com/auth/adwords%20https://www.googleapis.com/auth/bigquery

2. 使用 Project Owner 創建服務帳號

3. 執行示例代碼：

def run_quickstart():
    from google.cloud import bigquery_datatransfer_v1
    from google.protobuf.struct_pb2 import Struct

    client = bigquery_datatransfer_v1.DataTransferServiceClient()

    project = <project-id>
    parent = client.location_path(project, 'us')
    params = Struct()
    params.update({
        "customer_id": <customer-id>
    })

    transfer_config = {
        "destination_dataset_id": "test",
        "display_name": "test",
        "data_source_id": "adwords",
        "params": params
    }

    authorization_code = <authorization_code from step 1>
    response = client.create_transfer_config(parent, transfer_config, authorization_code)
    print(response)

if __name__ == '__main__':
    run_quickstart()

期望：數據傳輸創建但我得到錯誤：

Traceback (most recent call last):
  File "/project/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/project/lib/python3.7/site-packages/grpc/_channel.py", line 826, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/project/lib/python3.7/site-packages/grpc/_channel.py", line 729, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.INVALID_ARGUMENT
    details = "Request contains an invalid argument."
    debug_error_string = "{"created":"@1581747953.387292000","description":"Error received from peer ipv6:[2404:6800:4012:1::200a]:443","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Request contains an invalid argument.","grpc_status":3}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 49, in <module>
    run_quickstart()
  File "app.py", line 40, in run_quickstart
    parent, transfer_config, authorization_code)
  File "/project/lib/python3.7/site-packages/google/cloud/bigquery_datatransfer_v1/gapic/data_transfer_service_client.py", line 563, in create_transfer_config
    request, retry=retry, timeout=timeout, metadata=metadata
  File "/project/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py", line 143, in __call__
    return wrapped_func(*args, **kwargs)
  File "/project/lib/python3.7/site-packages/google/api_core/retry.py", line 286, in retry_wrapped_func
    on_error=on_error,
  File "/project/lib/python3.7/site-packages/google/api_core/retry.py", line 184, in retry_target
    return target()
  File "/project/lib/python3.7/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout
    return func(*args, **kwargs)
  File "/project/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
google.api_core.exceptions.InvalidArgument: 400 Request contains an invalid argument.

我找不到哪一步是錯誤的。 請幫助我弄清楚。

--- 更新 02-19

@mk_sta

我刪除了authorization_code然后 API 可以創建傳輸，但傳輸無法工作。 它會得到一個像這個截圖一樣的錯誤。

我認為錯誤是由authorization_code用於連接到谷歌廣告引起的。 你的轉移工作正常嗎？

參考：

• https://cloud.google.com/bigquery-transfer/docs/reference/datatransfer/rest/v1/projects.locations.transferConfigs/create

Answer 1

修改您的代碼片段后，我注意到一些要點需要提及：

1. 根據文檔頁面BigQuery Data Transfer Service需要初始服務帳戶擁有bigquery.admin角色才能繼承bigquery.datasets.update權限：

所有將創建轉移的用戶都必須被授予bigquery.admin預定義的 Cloud IAM 角色。 bigquery.admin 角色包括以下 BigQuery Data Transfer Service 權限：

bigquery.transfers.update

bigquery.transfers.get

2. 每當您調用create_transfer_config函數時， authorization_code是一個可選參數，可能會被跳過，根據特定的用戶情況啟動不同的身份驗證模型以進行傳輸配置。

我嘗試使用與我的 GCP 項目關聯的服務帳戶密鑰執行上述代碼：

export GOOGLE_APPLICATION_CREDENTIALS="<some_path>/[FILE_NAME].json"

我對源代碼進行了一些調整，以去除傳輸配置中的authorization_code ：

def run_quickstart():
    from google.cloud import bigquery_datatransfer_v1
    from google.protobuf.struct_pb2 import Struct

    client = bigquery_datatransfer_v1.DataTransferServiceClient()

    project = <project-id>
    parent = client.location_path(project, 'us')
    params = Struct()
    params.update({
        "customer_id": <customer-id>
    })

    transfer_config = {
        "destination_dataset_id": "test",
        "display_name": "test",
        "data_source_id": "adwords",
        "params": params
    }

    response = client.create_transfer_config(parent, transfer_config)
    print(response)

if __name__ == '__main__':
    run_quickstart()

這對我有用，我已成功收到響應正文：

name: "projects/.../..."
destination_dataset_id: "test"
display_name: "test"
update_time {
  seconds: 1582044242
  nanos: 441442000
}
data_source_id: "adwords"
next_run_time {
  seconds: 1581897600
}
params {
  fields {
    key: "customer_id"
    value {
      string_value: "XXXX"
    }
  }
}
user_id: <user_id>
dataset_region: "us"