[英]Can we change location from US to other region while reading data from Bigquery using Bigquery java library?
I am trying to read data from Bigquery using Bigquery java library .我正在尝试使用Bigquery java 库从Bigquery读取数据。
My dataset is not in US location, so when i am giving my dataset name to library , it is throwing an error that dataset not found in US location because it searches by default in US location.我的数据集不在美国位置,所以当我将我的数据集名称提供给 library 时,它会抛出一个错误,即在美国位置找不到数据集,因为它默认在美国位置搜索。
I have also tried giving the location using setLocation("asia-southeast1") but still it is finding in US location.我也尝试使用 setLocation("asia-southeast1") 给出位置,但它仍然在美国位置找到。
This is my code snippet:这是我的代码片段:
val bigquery: BigQuery =BigQueryOptions.newBuilder().setLocation("asia-southeast1").build().getService
val query = "SELECT TO_JSON_STRING(t, true) AS json_row FROM "+dbName+"."+tableName+" AS t"
logger.info("Query is " + query)
val queryResult: QueryJobConfiguration = QueryJobConfiguration.newBuilder(query).build
val result: TableResult = bigquery.query(queryResult)
I am writing code in SCALA.我正在 SCALA 中编写代码。 As it uses same libraries as JAVA and JAVA is more popular, thats why I am asking this for JAVA.
因为它使用与 JAVA 相同的库,而且 JAVA 更受欢迎,这就是为什么我要求 JAVA 这样做。
Please help me to know that how I can change location from US to southeast.请帮助我知道如何将位置从美国更改为东南部。
Can I change something inside QueryJobConfiguration as i have searched a-lot but i am unable to find anything.我是否可以更改QueryJobConfiguration 中的某些内容,因为我已经搜索了很多但我找不到任何东西。
My only requirement is that I want final result as TableResult.我唯一的要求是我希望最终结果为 TableResult。
This is the exception being thrown这是抛出的异常
com.google.cloud.bigquery.BigQueryException: Not found: Dataset XXXXXXXX was not found in location US
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:106)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.getQueryResults(HttpBigQueryRpc.java:584)
at com.google.cloud.bigquery.BigQueryImpl$34.call(BigQueryImpl.java:1203)
at com.google.cloud.bigquery.BigQueryImpl$34.call(BigQueryImpl.java:1198)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1197)
at com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1181)
at com.google.cloud.bigquery.Job$1.call(Job.java:329)
at com.google.cloud.bigquery.Job$1.call(Job.java:326)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.poll(RetryHelper.java:64)
at com.google.cloud.bigquery.Job.waitForQueryResults(Job.java:325)
at com.google.cloud.bigquery.Job.getQueryResults(Job.java:291)
at com.google.cloud.bigquery.BigQueryImpl.query(BigQueryImpl.java:1168)
...
Thanks in advance.提前致谢。
You shouldn't actually need to specify the location because BigQuery will infer it from the dataset being referenced in your query.您实际上不需要指定位置,因为 BigQuery 会从查询中引用的数据集推断出它。 See here .
见这里。
When loading data, querying data, or exporting data, BigQuery determines the location to run the job based on the datasets referenced in the request.
在加载数据、查询数据或导出数据时,BigQuery 会根据请求中引用的数据集确定运行作业的位置。 For example, if a query references a table in a dataset stored in the asia-northeast1 region, the query job will run in that region.
例如,如果查询引用存储在 asia-northeast1 区域中的数据集中的表,则查询作业将在该区域中运行。
I just tested using the Java SDK on a dataset/table I created in asia-southeast1
, and it worked without needing to explicitly specify the location.我刚刚在
asia-southeast1
创建的数据集/表上使用 Java SDK 进行了测试,它无需明确指定位置即可工作。
If it's still not working for you by default (check the table you're referncing actually exists), then you can specify the location by setting it in the JobId
and passing that to the overloaded method:如果默认情况下它仍然不适合您(检查您引用的表是否实际存在),那么您可以通过在
JobId
设置它并将其传递给重载方法来指定位置:
String query = "SELECT * FROM `grey-sort-challenge.asia_southeast1.a_table`;";
QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query)
.setUseLegacySql(Boolean.FALSE)
.build();
JobId id = JobId.newBuilder().setLocation("asia-southeast1")
.setRandomJob()
.build();
try {
for (FieldValueList row : BIGQUERY.query(queryConfig, id).iterateAll()) {
for (FieldValue val : row) {
System.out.printf("%s,", val.toString());
}
System.out.printf("\n");
}
} catch (InterruptedException e) {
e.printStackTrace();
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.