无法使用自定义值升级 airflow helm chart

Question

我在 Azure 中有一个集群，带有官方 airflow 掌舵图。 Helm chart 安装很好，没有问题。 当我尝试使用对掌舵“values.yaml”的更改来升级图表时，唯一的更改是尝试添加现有的 postgres 连接。 我遇到了问题。

我遵循https://airflow.apache.org/docs/helm-chart/stable/production-guide.html的指南

抱歉，我是 airflow 和 kube.netes 的新手，我正在学习 go，如有任何帮助，我们将不胜感激。 最下面是现在的yaml，很长，抱歉。

迁移 pod 日志

$ k logs airflow-run-airflow-migrations-lnb7n -n airflow

    DB: postgresql+psycopg2://airflow_user:***@hostname-vm.com:5432/airflow_test
    Performing upgrade with database postgresql+psycopg2://airflow_user:***@hostname-    vm.com:5432/airflow_test
    [2022-11-15T16:00:05.411+0000] {migration.py:204} INFO - Context impl PostgresqlImpl.
    [2022-11-15T16:00:05.411+0000] {migration.py:211} INFO - Will assume transactional DDL.
    [2022-11-15T16:00:05.467+0000] {db.py:1531} INFO - Creating tables
    INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
    INFO  [alembic.runtime.migration] Will assume transactional DDL.
    Traceback (most recent call last):
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/base.py", line 233, in _catch_revision_errors
    yield
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/base.py", line 443, in _upgrade_revs
    for script in reversed(list(revs))
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/revision.py", line 804, in iterate_revisions
    assert_relative_length=assert_relative_length,
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/revision.py", line 1454, in _collect_upgrade_revisions
    current_revisions = self.get_revisions(lower)
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/revision.py", line 527, in get_revisions
    return sum([self.get_revisions(id_elem) for id_elem in id_], ())
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/revision.py", line 527, in <listcomp>
    return sum([self.get_revisions(id_elem) for id_elem in id_], ())
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/revision.py", line 554, in get_revisions
    for rev_id in resolved_id
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/revision.py", line 554, in <genexpr>
    for rev_id in resolved_id
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/revision.py", line 635, in _revision_for_ident
    resolved_id,
    alembic.script.revision.ResolutionError: No such revision or branch 'b0d31815b5a6'

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
    File "/home/airflow/.local/bin/airflow", line 8, in <module>
    sys.exit(main())
    File "/home/airflow/.local/lib/python3.7/site-packages/airflow/__main__.py", line 39, in main
    args.func(args)
    File "/home/airflow/.local/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 52,   in command
    return func(*args, **kwargs)
    File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/cli.py", line 99, in wrapper
    return f(*args, **kwargs)
    File "/home/airflow/.local/lib/python3.7/site-packages/airflow/cli/commands/db_command.py", line 84, in upgradedb
    db.upgradedb(to_revision=to_revision, from_revision=from_revision,   show_sql_only=args.show_sql_only)
    File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/session.py", line 75, in wrapper
    return func(*args, session=session, **kwargs)
    File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/db.py", line 1532, in   upgradedb
    command.upgrade(config, revision=to_revision or 'heads')
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/command.py", line 322, in  upgrade
    script.run_env()
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/base.py", line 569, in   run_env
    util.load_python_file(self.dir, "env.py")
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 94, in   load_python_file
    module = load_module_py(module_id, path)
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 110, in   load_module_py
    spec.loader.exec_module(module)  # type: ignore
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/home/airflow/.local/lib/python3.7/site-packages/airflow/migrations/env.py", line 112,  in <module>
    run_migrations_online()
    File "/home/airflow/.local/lib/python3.7/site-packages/airflow/migrations/env.py", line 106,       in run_migrations_online
    context.run_migrations()
    File "<string>", line 8, in run_migrations
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/runtime/environment.py", line   853, in run_migrations
    self.get_context().run_migrations(**kw)
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/runtime/migration.py", line 611, in run_migrations
    for step in self._migrations_fn(heads, self):
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/command.py", line 311, in upgrade
    return script._upgrade_revs(revision, rev)
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/base.py", line 443, in _upgrade_revs
    for script in reversed(list(revs))
    File "/usr/local/lib/python3.7/contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
    File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/base.py", line 265, in  _catch_revision_errors
    raise util.CommandError(resolution) from re
    alembic.util.exc.CommandError: Can't locate revision identified by 'b0d31815b5a6'

helm 升级超时后 kubectl 报错：

$ helm upgrade --install airflow apache-airflow/airflow -n airflow -f values.yaml --debug
    warning: Upgrade "airflow" failed: post-upgrade hooks failed: job failed: BackoffLimitExceeded
    Error: UPGRADE FAILED: post-upgrade hooks failed: job failed: BackoffLimitExceeded
    helm.go:84: [debug] post-upgrade hooks failed: job failed: BackoffLimitExceeded
    UPGRADE FAILED
    main.newUpgradeCmd.func2
        helm.sh/helm/v3/cmd/helm/upgrade.go:201
    github.com/spf13/cobra.(*Command).execute
        github.com/spf13/cobra@v1.5.0/command.go:872
    github.com/spf13/cobra.(*Command).ExecuteC
        github.com/spf13/cobra@v1.5.0/command.go:990
    github.com/spf13/cobra.(*Command).Execute
        github.com/spf13/cobra@v1.5.0/command.go:918
    main.main
        helm.sh/helm/v3/cmd/helm/helm.go:83
    runtime.main
        runtime/proc.go:250
    runtime.goexit
        runtime/asm_arm64.s:1172

yaml 更改——配置映射的第 347 行、第 319-321 行——如果需要也可以发布。

我已经尝试了一个配置 map，并且只是从 helm 编辑了 values.yaml 中的字段。 我尝试了 postgres-psycopg2 和 postgres 作为协议。 对于主机名，我尝试了 vm-hostname.com 的名称以及 vm 的 ip。 postgres sql 设置为 false。 我还创建了一个功能更强大的新集群，但运气不佳。

我确实找到了以下链接，但这些都不适合我。

Airflow - 为什么外部数据库配置会破坏 helm 升级？

我确实只创建了一个可以连接到 postrges 数据库的基本 web/scheduler 容器。 我必须升级数据库才能连接。 在尝试升级掌舵图之前，我还重置了数据库。

    metadataConnection: 
        user: airflow_user
        pass: password
        protocol: postgresql+psycopg2
        host: vm-hostname.com
        port: 5432
        db: airflow_test
        sslmode: disable

Answer 1

我通过更改 values.yaml 中的 airflow 版本解决了这个问题

https://airflow.apache.org/docs/apache-airflow/stable/migrations-ref.html

File "/home/airflow/.local/lib/python3.7/site-packages/alembic/script/base.py", line 265, in  _catch_revision_errors
raise util.CommandError(resolution) from re
alembic.util.exc.CommandError: Can't locate revision identified by 'b0d31815b5a6'

修订错误引用 airflow 版本 2.4.2； 我试图使用 2.4.0。 愚蠢的错误。 希望这对某人有帮助。