[英]Dataprep - Dataflow fails when output is BigQuery
As a part of the POC I was trying to setup some data quality checks through Dataprep. 作为POC的一部分,我试图通过Dataprep进行一些数据质量检查。 There is a BigQuery table as a source and it should run a job with output to another BigQuery. 有一个BigQuery表作为源,它应该运行一个作业,并输出到另一个BigQuery。 Unfortunately that job fails with error: 不幸的是,该作业失败并出现错误:
java.lang.RuntimeException: Failed to create job with prefix beam_load_[thenameofthejob], reached max retries: 3, last failed job: null.
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob.runJob(BigQueryHelpers.java:196)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager.waitForDone(BigQueryHelpers.java:149)
at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.finishBundle(WriteTables.java:255) .
Do you have any hints how to solve this, please? 请问您如何解决这个问题?
I have edited recipes and did not setup any transformation there just to see if the job runs - it failed again. 我已经编辑了配方,并没有在那里设置任何转换,只是为了查看作业是否运行-再次失败。 It works when I output this dataprep dataflow into csv. 当我将此dataprep数据流输出到csv中时,它起作用。 It is all running in EU region. 它全部在欧盟地区运行。
try to go to the bigquery page and then go to task history. 尝试转到bigquery页面,然后转到任务历史记录。 the flollowing step is to go to the tab project history inside task history. 后续步骤是转到任务历史记录内的选项卡项目历史记录。 there you will find the red exclemation mark at the failed attempt. 尝试失败时,您会在此处找到红色的感叹号。 if you click on that case, you will see the specific error message for that case. 如果单击该案例,将看到该案例的特定错误消息。
in my case it was the locations that were different, but yours were fine so this is as far as i can help i guess. 就我而言,这是不同的位置,但是您的位置很好,所以据我所能估计。 hope it will help you. 希望对您有帮助。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.