如何在 BashOperator 执行的 python 脚本文件中使用 Xcom

Question

I am executing a python script file using a Bash Operator as following:

op = BashOperator(task_id='python',
                  bash_command='python3 py_script.py')

In the python file, I want to send a value to xcom if a condition is met in a loop, something like:

for i in range(10):
    if i == 2:
        print(i)
        xcom_push('hello')

Since it is in a loop and I want it to continue after i reach 2, I can't use a return . I have tried with:

from airflow.models import XCom

context['ti'].xcom_push('hello')
kwargs['ti'].xcom_push('hello')

But in both cases I receive an error like NameError: name 'kwargs' is not defined

What would be the solution? Thanks

Answer 1

If you set xcom_push=True in BashOperator then the last line written to stdout by the bash command will be pushed to an XCom ( docs ).

So, if after print(i) in your py_script.py no other lines are printed then the string representation of i will be pushed to an XCom for the pythonk task.