We are developing our ETL on airflow 1.10.6 using cloud composer on GCP. Everything works well but we keep getting error message in the logs and in the interface regarding relative imports unless every tasks are running properly
ERROR - Failed to import: /home/airflow/gcs/dags/filepath.py attempted relative import with no known parent package
Other files with the same architecture and the same imports do not create the same errors. It fails in the process_file
when running the command imp.load_source
. If we change to non relative imports it works well also.
Do you have an idea of which file is executed in this process_file
and how we could solve this error (only in the logging but not in the execution).
Thanks for your help
So We have found out that airflow for an obscure reason was considering it as a dag and trying to import it as a dag. We have added the folder of this file to .airflowignore
file and it has solved the issue
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.