简体   繁体   English

java bigquery存储写api

[英]java bigquery storage write api

Using the java bigquery storage api as documented here https:\/\/cloud.google.com\/bigquery\/docs\/write-api<\/a> .使用此处记录的 java bigquery 存储 api https:\/\/cloud.google.com\/bigquery\/docs\/write-api<\/a> 。 Keeping the write stream long lived and refreshing it when one of the non-retry-able errors happened as per thishttps:\/\/cloud.google.com\/bigquery\/docs\/write-api#error_handling<\/a>根据此https:\/\/cloud.google.com\/bigquery\/docs\/write-api#error_handling<\/a>在发生不可重试错误之一时,保持写入流长期存在并刷新它

I am sticking with default stream.我坚持使用默认流。 I have two tables and different parts of code responsible for writing to each table, maintaining its own stream writer.我有两个表和负责写入每个表的代码的不同部分,维护自己的流编写器。

If data is flowing, everything is fine.如果数据在流动,一切都很好。 No errors.没有错误。 However I want to test refreshing the stream writers work too so I wait for default stream timeout (10mins) which closes the stream and try writing again.但是我也想测试刷新流编写器的工作,所以我等待默认流超时(10 分钟)关闭流并再次尝试写入。 I can create the stream fine, no error there, but for one of the table I keep getting cancelled error wrapped in a Pre condition failed making my code refresh again and again.我可以很好地创建流,那里没有错误,但是对于其中一个表,我不断收到被取消的错误,包含在前提条件中失败,使我的代码一次又一次地刷新。

Original error because stream closed due to inactivity原始错误,因为流因不活动而关闭

! io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Stream is closed due to com.google.api.gax.rpc.AbortedException: io.grpc.StatusRuntimeException: ABORTED: Closing the stream because it has been inactive for 600 seconds. Entity: projects/<id>/datasets/<id>/tables/<id>/_default
! at com.google.cloud.bigquery.storage.v1beta2.StreamWriterV2.appendInternal(StreamWriterV2.java:263)
! at com.google.cloud.bigquery.storage.v1beta2.StreamWriterV2.append(StreamWriterV2.java:234)
! at com.google.cloud.bigquery.storage.v1beta2.JsonStreamWriter.append(JsonStreamWriter.java:114)
! at com.google.cloud.bigquery.storage.v1beta2.JsonStreamWriter.append(JsonStreamWriter.java:89)

could you try updating to using the v1 version of the client?您可以尝试更新为使用客户端的 v1 版本吗? We have added reconnect support for the JsonStreamwriter which should help with retries.我们添加了对 JsonStreamwriter 的重新连接支持,这应该有助于重试。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM