GCP Dataflow There was a problem refreshing your credentials

Question

I'm trying to create a dataflow in Java with Gradle. I've created a gradle task to upload the job to GCP via the DataflowRunner and set my credentials via environment "GOOGLE_APPLICATION_CREDENTIALS", "....json" , but when I view the job, all I see is:

Workflow failed. Causes: There was a problem refreshing your credentials. Please check: 
1. Dataflow API is enabled for your project. 
2. There is a robot service account for your project: 
service-[project number]@dataflow-service-producer-prod.iam.gserviceaccount.com 
should have access to your project. 
If this account does not appear in the permissions tab for your project, contact Dataflow support.

I've already made sure the dataflow API is enabled for my project and that the service account exists and has editor and Cloud Dataflow Service Agent roles. I've tested with my code and the sample code on the getting started page, same issue.

Answer 1

When you first use Dataflow in a new GCP project, you need to enable the API (Step 3 in the Quickstart), it could take few minutes. During this step a couple of service accounts are created:

Cloud Dataflow service account: service-<project-number>@dataflow-service-producer-prod.iam.gserviceaccount.com .

Controller service account: <project-number>-compute@developer.gserviceaccount.com .

The first account is the one in the error. I would think that you have not enabled the Dataflow API or you executed the job while the API was being enabled. As you mentioned you verified this, the nex step is checking if the default service accounts were modified, let's say, they were recreated or the default permissions changed. These revisions also apply for the Controller service account.

In addition, please don't forget to assign the proper permissions to the account that execute the Dataflow job.

UPDATE:

If the issue persists, it is probable that the service accounts were corrupted somehow, in which case, it is recommended to create a new GCP project with fresh service accounts or Contact Cloud Support .

Answer 2

This issue can be triggerd if the Compute Engine default service account in the Google Cloud project is disabled:

The status should show a green tick.