如何在运行后删除所有 Xcom？

Question

I am kind of new to airflow and I can not seem to find a way to delete all my Xcom after a run. I found the bash command airflow clear, but this deleted task instances as well and I just want the Xcom cleared. Any help is welcomed. Thanks!

Update:

clear_xcom_operator = SqliteOperator(
    task_id="clear_Xcom",
    sqlite_conn_id="sqlite_default",
    sql="delete from xcom where dag_id='dumper'",
    retries=0,
    dag=dag,
)

I am doing this but I get the error:

sqlite3.OperationalError: no such table: xcom

Any ideas?

Answer 1

I am not aware of a functionality to clear XCOM that is provided out of the box. Probably, the only ways is to manipulate the backend database directly, by deleting the entries in the xcom table based on dag_id .

XCOM is not meant to exchange large amounts of data between tasks. So, if space considerations is the reason for deleting XCOM values, a better solution would be to avoid using XCOM and write out the data to a shared storage.