Drain GCP dataflow jobs with terraform

Question

When deploying a dataflow job with terraform it does not drain the old job. Is there a possibility to automatically drain it and the deploy?

Answer 1

I think it's not the responsability of Terraform to deploy custom Dataflow jobs (not Dataflow templates).

It's more the responsability of the deployment task proposed by a CI CD pipeline.

In this case there is no possibility to automatically drain and deploy a new version of the job.

You have to develop your own script ( Shell for example) to do that and apply your strategy.

For example, a Dataflow job can be drained by a gcloud command:

gcloud dataflow jobs drain JOB_ID