[英]where to write DAG files in apache air flow?
I just started learning apache airflow, and I created an environment in composer in gcp and web server is working fine and everything, but I was just confused about where to write the DAG file?我刚开始学习 apache airflow,我在 gcp 和 web 的 composer 中创建了一个环境,服务器工作正常,一切正常,但我只是对在哪里编写 DAG 文件感到困惑? I mean I want to write the file where I can test it multiple times because in the web UI it's showing me a bucket where I can store the file, but I am unable to understand where to write the code.
我的意思是我想编写可以多次测试的文件,因为在 web UI 中,它向我显示了一个存储桶,我可以在其中存储文件,但我无法理解在哪里编写代码。 do I have to install airflow in my machine?
我必须在我的机器上安装 airflow 吗?
ps - i know this is a stupid question, any help will be appreciated ps - 我知道这是一个愚蠢的问题,任何帮助将不胜感激
You can install it locally yes.您可以在本地安装它是的。 If you want to test it locally this is the only way I think.
如果您想在本地进行测试,这是我认为的唯一方法。
There are couple of tools that you could do that - there is an astro
CLI for managing your "dag development" environment which is published by Astronomer, https://github.com/astronomer/astro-cli您可以使用几种工具 - 有一个
astro
CLI 用于管理您的“dag 开发”环境,它由 Astronomer 发布, https://github.com/astronomer/astro-cli
Also MWAA has their own tool too - I think, I think Composer has no Composer-specific one. MWAA 也有自己的工具——我想,我认为 Composer 没有特定于 Composer 的工具。
However for "generic" Airflow (which should be enough to start), you can use the community managed quick-start (either with local venv or Docker-Compose):但是对于“通用”Airflow(应该足以启动),您可以使用社区管理的快速启动(使用本地 venv 或 Docker-Compose):
https://airflow.apache.org/docs/apache-airflow/stable/start/index.html https://airflow.apache.org/docs/apache-airflow/stable/start/index.html
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.