[英]GCP Dataflow Computation Graph and Job Execution
Hi Everyone I tried hard to understand what is happening when I create a custom template in Google cloud Dataflow but failed to understand.大家好,当我在谷歌云数据流中创建自定义模板时,我努力了解发生了什么,但未能理解。 Thanks to GCP documentations.
感谢 GCP 文档。 Below is what I am achieving.
以下是我正在实现的目标。
I successfully created the template and I am able to execute the job.我成功创建了模板并且能够执行作业。 But I have following questions.
但我有以下问题。
Can anyone please share some information on it?任何人都可以分享一些信息吗?
Sources I referred and failed to get the answer: https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline#pipeline-lifecycle-from-pipeline-code-to-dataflow-job http://alumni.media.mit.edu/~wad/magiceight/isa/node3.html https://cloud.google.com/dataflow/docs/guides/setting-pipeline-options#configuring-pipelineoptions-for-local-execution https://beam.apache.org/documentation/basics/ https://beam.apache.org/documentation/runtime/model/ https://mehmandarov.com/apache-beam-pipeline-graph/我引用但未能得到答案的来源: https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline#pipeline-lifecycle-from-pipeline-code-to-dataflow-job http:/ /alumni.media.mit.edu/~wad/magiceight/isa/node3.html https://cloud.google.com/dataflow/docs/guides/setting-pipeline-options#configuring-pipelineoptions-for-local-execution https://beam.apache.org/documentation/basics/ https://beam.apache.org/documentation/runtime/model/ https://mehmandarov.com/apache-beam/-pipeline-graph
This depends on where the models are being loaded from.这取决于从哪里加载模型。 If they're loaded in the DoFns (most likely), then it will happen in the workers (during job execution).
如果它们加载到 DoFns 中(最有可能),那么它将发生在工作人员中(在作业执行期间)。
As for your other question, there should be no issues with multiple users triggering a template job simultaneously.至于您的其他问题,多个用户同时触发模板作业应该没有问题。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.