[英]How to run bash script file in Airflow
I have a bash script that creates a file (if it does not exist) that I want to run in Airflow, but when I try it fails.我有一个 bash 脚本,它创建了一个我想在 Airflow 中运行的文件(如果它不存在),但是当我尝试时它失败了。 How do I do this?
我该怎么做?
#!/bin/bash
#create_file.sh
file=filename.txt
if [ ! -e "$file" ] ; then
touch "$file"
fi
if [ ! -w "$file" ] ; then
echo cannot write to $file
exit 1
fi
and here's how I'm calling it in Airflow:这是我在 Airflow 中的调用方式:
create_command = """
./scripts/create_file.sh
"""
t1 = BashOperator(
task_id= 'create_file',
bash_command=create_command,
dag=dag
)
lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 83, in execute
raise AirflowException("Bash command failed")
airflow.exceptions.AirflowException: Bash command failed
From the tutorial this is OK:从教程这是可以的:
t2 = BashOperator(
task_id='sleep',
bash_command='sleep 5',
retries=3,
dag=dag)
But you're passing a multi-line command to it但是你向它传递了一个多行命令
create_command = """
./scripts/create_file.sh
"""
should be应该是
create_command = "./scripts/create_file.sh "
Moreover, you also have to make sure that you are in the correct directory to avoid cryptic errors.此外,您还必须确保您位于正确的目录中,以避免出现神秘错误。 Do it like this for example:
例如这样做:
create_command = "./scripts/create_file.sh "
if os.path.exists(create_command):
t1 = BashOperator(
task_id= 'create_file',
bash_command=create_command,
dag=dag
)
else:
raise Exception("Cannot locate {}".format(create_command))
From the documentation :从文档:
t2 = BashOperator(
task_id='bash_example',
# "scripts" folder is under "/usr/local/airflow/dags"
bash_command="scripts/test.sh",
dag=dag)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.