I am trying to build the connection between the postgres and airflow, here is my docker-compose
version: '3'
services:
postgres:
image: postgres
environment:
- POSTGRES_USER=airflow
- POSTGRES_PASSWORD=airflow
- POSTGRES_DB=airflow
webserver:
image: apache/airflow:1.10.13-python3.6
command: bash -c "airflow initdb; airflow webserver; airflow scheduler;"
environment:
- AIRFLOW_DATABASE_USERNAME=airflow
- AIRFLOW_DATABASE_PASSWORD=airflow
- AIRFLOW_DATABASE_NAME=airflow
volumes:
- ../airflow/dag:/opt/airflow/dags
ports:
- 8080:8080
After setting the postgres configuration setting it's still taking the SQLite as backend database. Logs as follow
ravi@ravi:~/Documents/airflow$ docker-compose up airflow_webserver
Starting airflow_airflow_webserver_1 ...
Starting airflow_airflow_webserver_1 ... done
Attaching to airflow_airflow_webserver_1
airflow_webserver_1 | DB: sqlite:////opt/airflow/airflow.db
airflow_webserver_1 | [2020-12-06 12:53:22,911] {db.py:378} INFO - Creating tables
airflow_webserver_1 | INFO [alembic.runtime.migration] Context impl SQLiteImpl.
airflow_webserver_1 | INFO [alembic.runtime.migration] Will assume non-transactional DDL.
airflow_webserver_1 | Done.
After adding the following two environments variable the airflow is now taking the backend database as Postgres
:
AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql+psycopg2://airflow:airflow@postgres/airflow
AIRFLOW__CORE__EXECUTOR=LocalExecutor
Thank you @elad
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.