airflow jinja2.exceptions.UndefinedError: 'airflow.models.taskinstance.TaskInstance object' 沒有屬性 'map_index'

Question

首先，我是 Airflow 和 Python 的新手。 過去，我在 wsl2 上安裝了 airflow 2.3.3（和一些提供程序）。 我的 dag 按預期工作，沒有任何錯誤我的 dag：

import os
from datetime import datetime
import logging
from airflow import models
from airflow.operators.bash import BashOperator
from airflow.providers.google.cloud.operators.bigquery import (
    BigQueryCreateEmptyDatasetOperator,
    BigQueryCreateEmptyTableOperator,
    BigQueryDeleteDatasetOperator,
    BigQueryDeleteTableOperator,
    BigQueryGetDatasetTablesOperator,
    BigQueryUpdateDatasetOperator,
    BigQueryUpdateTableOperator,
    BigQueryUpdateTableSchemaOperator,
    BigQueryUpsertTableOperator,
    BigQueryInsertJobOperator,
)
from airflow.utils.trigger_rule import TriggerRule

DAG_ID = "bigquery_dataset"
DATASET_NAME = f"dataset_{DAG_ID}"

with models.DAG(
    DAG_ID,
    schedule_interval="@once",
    start_date=datetime(2022, 8, 16),
    catchup=False,
    tags=["example", "bigquery"],
) as dag:

    create_dataset = BigQueryCreateEmptyDatasetOperator(task_id="create_dataset", dataset_id=DATASET_NAME, exists_ok=True)
    
    create_dataset

現在我必須將 airflow 從 2.3.3 降級到 2.2.5

腳步：

• 完全卸載 airflow
• 安裝特定版本的 airflow
• 安裝與 airflow 兼容的提供程序

現在我重新創建並觸發了我的 dag，但我得到了錯誤：

 scheduler | [2022-08-19 15:31:05,927] {sequential_executor.py:59} INFO - Executing command: ['airflow', 'tasks', 'run', 'bigquery_dataset', 'create_view', 'manual__2022-08-19T08:30:58.710367+00:00', '--local', '--subdir', 'DAGS_FOLDER/test_bigquery.py']
 scheduler | [2022-08-19 15:31:07,217] {dagbag.py:500} INFO - Filling up the DagBag from /home/bao/airflow/dags/test_bigquery.py
 scheduler | Running <TaskInstance: bigquery_dataset.create_view manual__2022-08-19T08:30:58.710367+00:00 [queued]> on host DESKTOP-H8O5RAP.localdomain
 scheduler | Traceback (most recent call last):
 scheduler | File "/home/bao/.local/bin/airflow", line 8, in <module>
 scheduler | sys.exit(main())
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/__main__.py", line 48, in main
 scheduler | args.func(args)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command
 scheduler | return func(*args, **kwargs)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line 92, in wrapper
 scheduler | return f(*args, **kwargs)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py", line 298, in task_run
 scheduler | _run_task_by_selected_method(args, dag, ti)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py", line 105, in _run_task_by_selected_method
 scheduler | _run_task_by_local_task_job(args, ti)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py", line 163, in _run_task_by_local_task_job
 scheduler | run_job.run()
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/jobs/base_job.py", line 246, in run
 scheduler | self._execute()
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/jobs/local_task_job.py", line 78, in _execute
 scheduler | self.task_runner = get_task_runner(self)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/task/task_runner/__init__.py", line 63, in get_task_runner
 scheduler | task_runner = task_runner_class(local_task_job)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/task/task_runner/standard_task_runner.py", line 35, in __init__
 scheduler | super().__init__(local_task_job)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/task/task_runner/base_task_runner.py", line 48, in __init__
 scheduler | super().__init__(local_task_job.task_instance)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/utils/log/logging_mixin.py", line 40, in __init__
 scheduler | self._set_context(context)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/utils/log/logging_mixin.py", line 54, in _set_context
 scheduler | set_context(self.log, context)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/utils/log/logging_mixin.py", line 178, in set_context
 scheduler | handler.set_context(value)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/utils/log/file_task_handler.py", line 60, in set_context
 scheduler | local_loc = self._init_file(ti)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/utils/log/file_task_handler.py", line 283, in _init_file
 scheduler | relative_path = self._render_filename(ti, ti.try_number)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/utils/log/file_task_handler.py", line 85, in _render_filename
 scheduler | return render_template_to_string(self.filename_jinja_template, context)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/utils/helpers.py", line 268, in render_template_to_string
 scheduler | return render_template(template, context, native=False)
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/airflow/utils/helpers.py", line 263, in render_template
 scheduler | return "".join(nodes)
 scheduler | File "<template>", line 20, in root
 scheduler | File "/home/bao/.local/lib/python3.8/site-packages/jinja2/runtime.py", line 903, in _fail_with_undefined_error
 scheduler | raise self._undefined_exception(self._undefined_message)
 scheduler | jinja2.exceptions.UndefinedError: 'airflow.models.taskinstance.TaskInstance object' has no attribute 'map_index'
 scheduler | [2022-08-19 15:31:08,234] {sequential_executor.py:66} ERROR - Failed to execute task Command '['airflow', 'tasks', 'run', 'bigquery_dataset', 'create_view', 'manual__2022-08-19T08:30:58.710367+00:00', '--local', '--subdir', 'DAGS_FOLDER/test_bigquery.py']' returned non-zero exit status 1..

日志說 jinja2 嘗試渲染任務實例但由於沒有屬性“map_index”而失敗，對吧？ 我的代碼（或我的環境）是否存在版本沖突或問題？ 謝謝

Answer 1

看起來您的數據庫架構已過時。 或者，實際上，數據庫是正確的，但 airflow 是錯誤的。

嘗試檢查 airflow 版本並遷移數據庫或重新安裝 airflow：

airflow version

airflow db update  # If airflow version is correct
pip3 install airflow==2.2.5  # If not

Idk，這個命令可以降級數據庫，但如果你從 2.2.5 遷移到 2.3.4，它肯定會有所幫助。

More about it on https://airflow.apache.org/docs/apache-airflow/stable/installation/upgrading.html , https://airflow.apache.org/docs/apache-airflow/stable/migrations-ref. html和https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html#db