使用 fs.createWriteStream 将数据写入 bigquery (node.js) 时出现模式错误

Question

错误： No schema specified on job or table 。

不知道为什么会发生此错误。 代码来自文档。 我也尝试过使用不同的格式，例如 fs.createWriteStream({sourceFormat: "json"}) - 但它会导致相同的错误。

const { BigQuery } = require("@google-cloud/bigquery");
const bigquery = new BigQuery();
const dataset = bigquery.dataset("firebase_test_data");
const table = dataset.table("flattened_data");
const fs = require("fs");

fs.createReadStream("./data.json")
  .pipe(table.createWriteStream("json"))
  .on("job", (job) => {
    // `job` is a Job object that can be used to check the status of the
    // request.
    console.log(job);
  })
  .on("complete", (job) => {
    // The job has completed successfully.
  });

Answer 1

您收到此错误是因为const table = dataset.table("flattened_data");中定义的表没有您在 data.json 中传递的适当架构。

您可以根据Google 文档尝试以下代码，并在 BigQuery 中指定表的架构，从而成功地将数据加载到表中。

const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const dataset = bigquery.dataset('my-dataset');
const table = dataset.table('my-table');

//-
// Load data from a JSON file.
//-
const fs = require('fs');

fs.createReadStream('/path-to-json/data.json')
 .pipe(table.createWriteStream('json'))
 .on('job', (job) => {
   // `job` is a Job object that can be used to check the status of the
   // request.
 })
 .on('complete', (job) => {
   // The job has completed successfully.
 });