如何使用 node.js 將 JSON 文件加載到 google Bigquery

Question

我正在使用 fetch 函數獲取附加的 JSON 對象，並使用我的 node.js 后端使用以下代碼將此 JSON 數據加載到 Bigquery：

await bigquery
      .dataset(datasetId)
      .table(tableId).insert(JSON_obj)

但不幸的是收到以下錯誤：

Unhandled rejection PartialFailureError: A failure occurred during this request

JSON對象

{
  "totalResults": 418,
  "profileInfo": {
    "profileId": "104881487",
    "profileName": "All Mobile App Data",
    "accountId": "64812694",
    "tableId": "ga:105536427",
    "internalWebPropertyId": "100521715",
    "webPropertyId": "UA-648333494-1"
  },
  "totalsForAllResults": {
    "ga:users": "427",
    "ga:totalEvents": "682",
    "ga:eventValue": "0"
  },
  "query": {
    "max-results": 1000,
    "start-index": 1,
    "start-date": "today",
    "end-date": "today",
    "dimensions": "ga:eventCategory,ga:eventAction,ga:eventLabel,ga:dateHourMinute",
    "metrics": [
      "ga:users",
      "ga:totalEvents",
      "ga:eventValue"
    ],
    "ids": "ga:104831427",
    "sort": [
      "-ga:totalEvents"
    ]
  },
  "selfLink": "https://www.googleapis.com/analytics/v3/data/ga?ids=ga:10483467&dimensions=ga:eventCategory,ga:eventAction,ga:eventLabel,ga:dateHourMinute&metrics=ga:users,ga:totalEvents,ga:eventValue&sort=-ga:totalEvents&start-date=today&end-date=today",
  "columnHeaders": [
    {
      "name": "ga:eventCategory",
      "columnType": "DIMENSION",
      "dataType": "STRING"
    },
    {
      "name": "ga:eventAction",
      "columnType": "DIMENSION",
      "dataType": "STRING"
    },
    {
      "name": "ga:eventLabel",
      "columnType": "DIMENSION",
      "dataType": "STRING"
    },
    {
      "name": "ga:dateHourMinute",
      "columnType": "DIMENSION",
      "dataType": "STRING"
    },
    {
      "name": "ga:users",
      "columnType": "METRIC",
      "dataType": "INTEGER"
    },
    {
      "name": "ga:totalEvents",
      "columnType": "METRIC",
      "dataType": "INTEGER"
    },
    {
      "name": "ga:eventValue",
      "columnType": "METRIC",
      "dataType": "INTEGER"
    }
  ],
  "containsSampledData": false,
  "id": "https://www.googleapis.com/analytics/v3/data/ga?ids=ga:104831427&dimensions=ga:eventCategory,ga:eventAction,ga:eventLabel,ga:dateHourMinute&metrics=ga:users,ga:totalEvents,ga:eventValue&sort=-ga:totalEvents&start-date=today&end-date=today",
  "itemsPerPage": 1000,
  "kind": "analytics#gaData",
  "rows": [
    [
      "video_screen",
      "click_on_screen",
      "false",
      "202011190517",
      "1",
      "32",
      "0"
    ],
    [
      "video_screen",
      "click_on_screen",
      "false",
      "202011190730",
      "1",
      "17",
      "0"
    ],
    ...

Answer 1

當您將 JSON 提交給 BigQuery 插入到表函數時，您只需提供所需的數據。 在這里，您提供了一個很大的 JSON，庫需要猜測數據才能進入其中。

我個人猜測rows數組中，但我不確定。 而且我也不確定現場順序！！

因此，從您的 JSON 中提取有用的數據，根據需要對其進行格式化（CSV、JSON 等）並將它們提交給 BigQuery。 它會更好地工作！