简体   繁体   中英

how to set Postgres as backend database in Airflow?

I am trying to build the connection between the postgres and airflow, here is my docker-compose

version: '3'
services:
    postgres:
        image: postgres
        environment:
            - POSTGRES_USER=airflow
            - POSTGRES_PASSWORD=airflow
            - POSTGRES_DB=airflow
    webserver:
        image: apache/airflow:1.10.13-python3.6
        command: bash -c "airflow initdb; airflow webserver; airflow scheduler;"
        environment:
            - AIRFLOW_DATABASE_USERNAME=airflow
            - AIRFLOW_DATABASE_PASSWORD=airflow
            - AIRFLOW_DATABASE_NAME=airflow
        volumes:
            - ../airflow/dag:/opt/airflow/dags
        ports:
           - 8080:8080

After setting the postgres configuration setting it's still taking the SQLite as backend database. Logs as follow

ravi@ravi:~/Documents/airflow$ docker-compose up airflow_webserver 
Starting airflow_airflow_webserver_1 ... 
Starting airflow_airflow_webserver_1 ... done
Attaching to airflow_airflow_webserver_1
airflow_webserver_1  | DB: sqlite:////opt/airflow/airflow.db
airflow_webserver_1  | [2020-12-06 12:53:22,911] {db.py:378} INFO - Creating tables
airflow_webserver_1  | INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
airflow_webserver_1  | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
airflow_webserver_1  | Done.

After adding the following two environments variable the airflow is now taking the backend database as Postgres :

AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql+psycopg2://airflow:airflow@postgres/airflow 
AIRFLOW__CORE__EXECUTOR=LocalExecutor

Thank you @elad

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM