简体   繁体   English

Composer/Airflow - 在运行 DAG/管道时添加依赖项?

[英]Composer/Airflow - Adding a dependency while running a DAG/pipeline?

this might be silly for many, but I did some research and couldn't find the answer, so that's why I'm asking:这对很多人来说可能很愚蠢,但我做了一些研究但找不到答案,所以这就是我问的原因:

  • When you're adding a new dependency, let's say Pandas, to you Composer Machine in GCP, does it mess up with the running of your pipelines?当您向 GCP 中的 Composer Machine 添加一个新的依赖项时,比如说 Pandas,它会扰乱您的管道运行吗? Because the 'server' kind of restarts and keep thinking for at least 5 mins when adding a new dependency.因为在添加新依赖项时,“服务器”类型会重新启动并至少思考 5 分钟。 Again, my question is: can this process of adding a dependency makes your DAGs fail or your pipelines crash or something like that?同样,我的问题是:这个添加依赖项的过程是否会使您的 DAG 失败或您的管道崩溃或类似的事情? Thanks!谢谢!

I believe this could mess up with your running DAGs.我相信这可能会扰乱您正在运行的 DAG。
When you add a new dependency, the environment must upgrade and you need to pause all DAGs and wait for in-progress tasks to finish before upgrading ( Reference ).添加新的依赖项时,环境必须升级,您需要暂停所有 DAG 并等待正在进行的任务完成才能升级( 参考资料)。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM