[英]How to set up emr_default in airflow
Dear Stackoverflow member, 尊敬的Stackoverflow成员,
I am trying to use Airflow to schedule a DAG that launch a cluster, run some spark jobs then terminated itself. 我正在尝试使用Airflow安排启动群集的DAG,运行一些Spark作业然后终止自身。 I tried to follow this example: https://insight.io/github.com/apache/incubator-airflow/blob/master/airflow/contrib/example_dags/example_emr_job_flow_automatic_steps.py 我试图按照以下示例操作: https : //insight.io/github.com/apache/incubator-airflow/blob/master/airflow/contrib/example_dags/example_emr_job_flow_automatic_steps.py
It gave me the error: airflow.exceptions.AirflowException: The conn_id emr_default
isn't defined 它给了我错误:airflow.exceptions.AirflowException: emr_default
I probably need to have emr_default
set up (please correct me if this is not the case). 我可能需要设置emr_default
(如果不是这种情况,请更正我)。 Does anyone have try this before and give me some hints on how to set it up? 有人以前尝试过此方法,并给我一些如何设置的提示吗? Airflow website doesn't provide too much on emr connection: http://pythonhosted.org/airflow/configuration.html?highlight=connection#connections Airflow网站在emr连接方面没有提供太多信息: http ://pythonhosted.org/airflow/configuration.html?highlight=connection#connections
Thanks! 谢谢!
在气流UI上,转到“管理”->“连接”选项卡,然后编辑emr_default行的值。
As@Him said, you can edit in airflow. 正如@Him所说,您可以编辑气流。 but if you can't find emr_default
, you can go to Admin
-> Connections
-> Create
to add any connection you want 但如果你不能找到emr_default
,你可以去Admin
- > Connections
- > Create
添加任何你想要的连接
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.