简体   繁体   English

如何在 Airflow 中将 Postgres 设置为后端数据库?

[英]how to set Postgres as backend database in Airflow?

I am trying to build the connection between the postgres and airflow, here is my docker-compose我正在尝试在 postgres 和 airflow 之间建立连接,这是我的 docker-compose

version: '3'
services:
    postgres:
        image: postgres
        environment:
            - POSTGRES_USER=airflow
            - POSTGRES_PASSWORD=airflow
            - POSTGRES_DB=airflow
    webserver:
        image: apache/airflow:1.10.13-python3.6
        command: bash -c "airflow initdb; airflow webserver; airflow scheduler;"
        environment:
            - AIRFLOW_DATABASE_USERNAME=airflow
            - AIRFLOW_DATABASE_PASSWORD=airflow
            - AIRFLOW_DATABASE_NAME=airflow
        volumes:
            - ../airflow/dag:/opt/airflow/dags
        ports:
           - 8080:8080

After setting the postgres configuration setting it's still taking the SQLite as backend database.设置 postgres 配置设置后,它仍然将 SQLite 作为后端数据库。 Logs as follow日志如下

ravi@ravi:~/Documents/airflow$ docker-compose up airflow_webserver 
Starting airflow_airflow_webserver_1 ... 
Starting airflow_airflow_webserver_1 ... done
Attaching to airflow_airflow_webserver_1
airflow_webserver_1  | DB: sqlite:////opt/airflow/airflow.db
airflow_webserver_1  | [2020-12-06 12:53:22,911] {db.py:378} INFO - Creating tables
airflow_webserver_1  | INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
airflow_webserver_1  | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
airflow_webserver_1  | Done.

After adding the following two environments variable the airflow is now taking the backend database as Postgres :添加以下两个环境变量后, airflow 现在将后端数据库作为Postgres

AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql+psycopg2://airflow:airflow@postgres/airflow 
AIRFLOW__CORE__EXECUTOR=LocalExecutor

Thank you @elad谢谢@elad

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 Apache Airflow 数据库后端需要哪些 postgres 权限? - What postgres privileges are required for Apache Airflow database backend? 如何在 airflow 上为 postgres 操作员设置 postgres 连接 - How do I set up a postgres connection on airflow for the postgres operator 如何将数据库(postgres)连接到谷歌云平台上的 Airflow composer? - How to Connect Database(postgres) to Airflow composer On Google Cloud Platform? 设置 Postgres celery result_backend 时气流调度程序崩溃 - Airflow Scheduler Crashes when setting Postgres celery result_backend 用于 Airflow 后端数据库的 Postgres 在指定模式中创建元数据表 - Postgres for Airflow backend DB to create metadata tables in a specified schema 使用气流的 postgres 数据库获取应用程序数据 - Using airflow's postgres database for application data Postgres“系统数据库”属性是如何设置或取消设置的? - How is the Postgres 'system database' attribute set or unset? 运行 airflow initdb 引发错误“postgres”以解决:使用 airflow 元数据库的 postgres 的名称或服务未知 - Running airflow initdb raises error `"postgres" to address: Name or service not known` using postgres for airflow meta database 如何在Postgres后端使用Cayley? - How to use Cayley with a Postgres backend? 配置 Apache Airflow 以使用 postgres 数据库时出错 - Errors when configuring Apache Airflow to use a postgres database
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM