[英]Airflow 2 Push Xcom with Key Name
In Airflow 2 taskflow API I can, using the following code examples, easily push and pull XCom values between tasks:-在 Airflow 2 taskflow API 我可以使用以下代码示例轻松地在任务之间推送和拉取 XCom 值:-
@task(task_id="task_one")
def get_height() -> int:
response = requests.get("https://swapi.dev/api/people/4")
data = json.loads(response.text)
height = int(data["height"])
return height
@task(task_id="task_two")
def check_height(val):
# Show val:
print(f"Value passed in is: {val}")
check_height(get_height())
I can see that the val passed into check_height is 202 and is wrapped in the xcom default key 'return_value' and that's fine for some of the time, but I generally prefer to use specific keys.我可以看到传递到 check_height 中的 val 是 202,并且包装在 xcom 默认键“return_value”中,这在某些时候没问题,但我通常更喜欢使用特定的键。
My question is how can I push the XCom with a named key?我的问题是如何使用命名密钥推送 XCom? This was really easy previously with ti.xcom_push where you could just supply the key name you wanted the value to be stuffed into, but I can't quite put my finger on how to achieve this in the taskflow api workflow.
以前使用 ti.xcom_push 时这真的很容易,您可以在其中提供您希望将值填充到其中的键名,但我不太清楚如何在任务流 api 工作流中实现这一点。
Would appreciate any pointers or (simple, please.) examples on how to do this.将不胜感激有关如何执行此操作的任何指示或(请简单。)示例。
You can just set ti
in the decorator as:您可以在装饰器中将
ti
设置为:
@task(task_id="task_one", ti)
def get_height() -> int:
response = requests.get("https://swapi.dev/api/people/4")
data = json.loads(response.text)
height = int(data["height"])
# Handle named Xcom
ti.xcom_push("my_key", height)
For cases where you need context in deep function you can also use get_current_context
.对于需要深度 function 上下文的情况,您还可以使用
get_current_context
。 I'll use it in my example below just to show it but it's not really required in your case.我将在下面的示例中使用它只是为了展示它,但在您的情况下并不是真正需要的。
here is a working example:这是一个工作示例:
import json
from datetime import datetime
import requests
from airflow.decorators import dag, task
from airflow.operators.python import get_current_context
DEFAULT_ARGS = {"owner": "airflow"}
@dag(dag_id="stackoverflow_dag", default_args=DEFAULT_ARGS, schedule_interval=None, start_date=datetime(2020, 2, 2))
def my_dag():
@task(task_id="task_one")
def get_height() -> int:
response = requests.get("https://swapi.dev/api/people/4")
data = json.loads(response.text)
height = int(data["height"])
# Handle named Xcom
context = get_current_context()
ti = context["ti"]
ti.xcom_push("my_key", height)
return height
@task(task_id="task_two")
def check_height(val):
# Show val:
print(f"Value passed in is: {val}")
#Read from named Xcom
context = get_current_context()
ti = context["ti"]
ti.xcom_pull("task_one")
print(f"Value passed from xcom my_key is: {val}")
check_height(get_height())
my_dag = my_dag()
two xcoms being pushed (one for the returned value and one with the by the key we choose):两个 xcom 被推送(一个用于返回值,一个用于我们选择的键):
printing the two xcoms in downstream task_two
:在下游
task_two
:
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.