Implement "WITH .. AS" Statements in Airflow Dags

Question

In Amazon Athena UI, I can do something like this:

WITH outer_query AS (SELECT id FROM outer)
SELECT * FROM inner JOIN ON inner.id = outer_query.id

How do I do this in Dags in Airflow using Python? The examples I saw uses simple queries using PythonOperator and doesn't use the WITH.. AS.. statement. Thank you!

Answer 1

You will want to use the AthenaOperator https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/operators/athena.html#run-a-query-in-amazon-athena

and you can use the WITH... AS syntax in Athena https://docs.aws.amazon.com/athena/latest/ug/select.html