Is it possible to sync an azure repo with MWAA (Amazon Workflows for Apache Airflow)?

Question

I have set up a private MWAA instance in AWS. It has set up a bucket that stores DAGs in S3. I've created a private repository in Azure DevOps and have set up a role that can access this bucket.

With Azure-Pipelines is it possible to sync the entire repository to control the DAGs created/modified in that S3 bucket?

I've seen it's possible to create artefacts and push them to the S3 bucket, but what if a dag is deleted? The DAG will still persist in the S3 Bucket and will still be available in MWAA.

Any guidance will be appreciated.

Answer 1

If you just want to sync entire repository to S3 bucket,you can use the task Amazon S3 Upload in your azure pipeline.

I'm not sure if that will fully address your problem, though.

If there is any misunderstanding, please feel free to add comments related to your issue.