[英]Use XCOM Value In Operators
I want to use XCOM values as a parameter of my Operator.我想使用 XCOM 值作为我的 Operator 的参数。
Firstly, was executed OracleReadOperator, which read table from db, and return values.首先,执行 OracleReadOperator,它从数据库中读取表,并返回值。
This is value in XCOM:这是 XCOM 中的价值:
[{'SOURCE_HOST': 'TEST_HOST'}]
Using this function I want to get value from xcom使用此功能我想从 xcom 获取价值
def print_xcom(**kwargs):
ti = kwargs['ti']
ti.xcom_pull(task_ids='task1')
Then use values as as parameter:然后使用值作为参数:
with DAG(
schedule_interval='@daily',
dagrun_timeout=timedelta(minutes=120),
default_args=args,
template_searchpath=tmpl_search_path,
catchup=False,
dag_id='test'
) as dag:
test_l = OracleLoadOperator(
task_id = "task1",
oracle_conn_id="orcl_conn_id",
object_name='table'
)
test_l
def print_xcom(**kwargs):
ti = kwargs['ti']
ti.xcom_pull(task_ids='task1', value='TARGET_TABLE')
load_from_db = MsSqlToOracleTransfer(
task_id= 'task2',
mssql_conn_id = "{task_instance.xcom_pull(task_ids='task1') }",
oracle_conn_id = 'conn_def_orc',
sql= 'test.sql',
oracle_table = "oracle_table"
tasks.append(load_from_db)
I don't know do I need print_xcom function.我不知道我是否需要 print_xcom 功能。 Or I can get value without it, if yes how?
或者我可以在没有它的情况下获得价值,如果是的话如何? I got this error:
我收到此错误:
airflow.exceptions.AirflowNotFoundException: The conn_id `{ task_instance.xcom_pull(task_ids='task1') }` isn't defined
To resolve the immediate
NameError
exception, Jinja expressions are strings so the arg for
oracle_table
needs to be updated to:
为了解决即时的
NameError
异常,Jinja 表达式是字符串,因此需要将
oracle_table
的 arg 更新为:
oracle_table = "{{ task_instance.xcom_pull(task_ids='print_xcom', key='task1') }}"
(Since the question and problem changed.) (因为问题和问题发生了变化。)
Only template_fields
declared for an operator can use Jinja expressions.只有为运算符声明的
template_fields
才能使用 Jinja 表达式。 It looks like MsSqlToOracleTransfer
is a custom operator and if you want to use a Jinja template for the mssql_conn_id
arg, it needs to be declared as part of template_fields
otherwise the literal string is used as the arg value (which is what you're seeing).看起来
MsSqlToOracleTransfer
是一个自定义运算符,如果您想对mssql_conn_id
arg 使用 Jinja 模板,则需要将其声明为template_fields
一部分,否则文字字符串将用作 arg 值(这就是您所看到的) . Also you need the expression in the "{{ ... }}"
format as well.您还需要
"{{ ... }}"
格式的表达式。
Here is some guidance on Jinja templating with custom operators if you find it helpful.如果您觉得有帮助, 这里有一些关于使用自定义运算符的 Jinja 模板的指南。
However, it seems like there is more to this picture than what we have context for.然而,这张图片似乎比我们所了解的更多。 What is
task1
?什么是
task1
? Are you simply trying to retrieve a connection ID?您是否只是想检索连接 ID? What is it exactly you are trying to accomplish accessing
XComs
in the DAG?究竟是什么,你要完成访问
XComs
在DAG?
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.