[英]Strreaming json Data into Bigquery using java.With out using a job to load data
I am new to bigquery so i do not fully understand how to stream data to bigquery, this is my problem, I have jsonInString, mapped from object like this 我是bigquery的新手,所以我不完全了解如何将数据流传输到bigquery,这是我的问题,我有jsonInString,是从这样的对象映射的
String customerJsonInString = mapper.writeValueAsString(customer);
{
"id": "1",
"first_name": "John",
"last_name": "Doe",
"dob": "1968-01-22",
"addresses": [
{
"status": "current",
"address": "123 First Avenue",
"city": "Seattle",
"state": "WA",
"zip": "11111",
"numberOfYears": "1"
},
{
"status": "previous",
"address": "456 Main Street",
"city": "Portland",
"state": "OR",
"zip": "22222",
"numberOfYears": "5"
}
]
}
the table has been created with the correct schema. 该表已使用正确的架构创建。 now i want to stream this data into bigquery (insert row),i am using the example that is on ( https://cloud.google.com/bigquery/streaming-data-into-bigquery#bigquery-stream-data-java ) to adapt it to mine, this is what i tried,
现在我想将此数据流式传输到bigquery(插入行)中,我使用的是以下示例( https://cloud.google.com/bigquery/streaming-data-into-bigquery#bigquery-stream-data-java )以使其适合我的,这就是我尝试过的,
TableId tableId = TableId.of(DATASET_NAME,TABLE_NAME);
Map<String, Object> recordsContent = new HashMap<>();
recordsContent.put("Customer", customerJsonInString);
InsertAllResponse response = bigquery.insertAll(InsertAllRequest.newBuilder(tableId)
.addRow("rowId", recordsContent)
.build());
if (response.hasErrors()) {
for (Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
}
}
I'm guessing the schema of the table is id, first_name, last_name, ...
instead of customer
? 我猜该表的架构是
id, first_name, last_name, ...
而不是customer
? In that case you'll need to set the fields one by one instead of as a whole json string, eg recordsContent.put("id", 1)
. 在这种情况下,您需要一个一个地设置字段,而不是整个json字符串,例如
recordsContent.put("id", 1)
。
Check the example with comments . 检查带有注释的示例 。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.