Creating pubsub subscription to topic failed with error: User not authorized to perform this action

Question

When I try to deploy a dataflow job with a service account, it fails with the following error:

Workflow failed. Causes: Step "setup_resource_XXXXXX.subscription-1210256155950706891639" failed., Step setup_resource_XXXXXX.subscription-1210256155950706891639: Set up of resource XXXXXX.subscription-12102561559507068916 failed, Creating pubsub subscription projects/XXXXXX/subscriptions/XXXXXX.subscription-12102561559507068916 to topic projects/XXXXXX/topics/XXXXXX failed with error: User not authorized to perform this action.

I have given the service account "Pub/Sub Subscriber" permission for that topic.

Answer 1

I faced the similar issue when I used PubSub to Cloud Storage (Text) template of dataflow job, I had to give pub/sub subscriber and pub/sub viewer roles to my controller service account of dataflow on pubsub topic and Storage Object Admin role on the bucket to the same controller service account of dataflow. It solved the issue for me.

Answer 2

It looks like Dataflow is trying to create a subscription here :

Creating pubsub subscription projects/XXXXXX/subscriptions/XXXXXX.subscription-12102561559507068916

However, role roles/pubsub.subscriber is not sufficient to create subscriptions, because it does not have pubsub.subscriptions.create permission.

You might need to grant your service account the role roles/pubsub.editor instead ( pubsub.subscriptions.create permissions is assigned to that role).

Answer 3

Following the suggestion of norbjb(Jun 3 '19 at 12:56), I have found that your Service Account need be granted:

Cloud Dataflow Service Agent

That solved the issue for me.