Cannot list service accounts on new GCE VM

Question

I created a new VM in a google compute engine project. I changed the "Compute Engine" access scope to "Read Write" after creating the VM.

On the existing (long-running) VM, if I do:

gcloud iam service-accounts list

I see the default service account for the project.

However, if I do the same thing on the newly created VM, I get an error:

  gcloud iam service-accounts list
ERROR: (gcloud.iam.service-accounts.list) User [<service-account>] does not have permission to access projects instance [<project>] (or it may not exist): Request had insufficient authentication scopes.

The original VM is a ubuntu-16, the new VM is ubuntu-18 freshly created from a google image.

If I look at the project IAM roles, my user has the following roles:

 - Access Approval Config Editor
 - Compute Admin
 - Role Viewer
 - Service Account Admin
 - Owner
 - Organization Administrator

What am I missing? The access scopes for the two VMs are the same:

 - Compute Engine               Read Write
 - Service Control              Enabled
 - Service Management           Read Only
 - Stackdriver Logging API      Write Only
 - Stackdriver Monitoring API   Write Only
 - Stackdriver Trace            Write Only
 - Storage                      Read Only

What controls access for the individual VMs other than the access scopes?

Answer 1

The problem was the SSH window was running under the service account, not my normal user account. I needed to run

gcloud init

to reconfigure to use my regular account.

I discovered this by doing

gcloud config list

on both machines.

Answer 2

PART 1

What controls access for the individual VMs other than the access scopes?

The union of Compute Engine Scopes and service account permissions.

Google Compute Engine scopes limit permissions, scopes do not grant permissions.

The service account assigned to Compute Engine determines the permissions/roles that are available. Scopes can limit those permissions granted to the service account. Scopes cannot grant permissions that the service account does not already have.

Scopes are a legacy authorization mechanism.

PART 2

gcloud iam service-accounts list ERROR: (gcloud.iam.service-accounts.list) User [] does not have permission to access projects instance [] (or it may not exist): Request had insufficient authentication scopes.

Part of this message is confusing to most people. Scopes are a legacy authentication mechanism that Google used before IAM. Scopes are similar to permissions and in this message mean OAuth 2 Permissions .

The command gcloud iam service-accounts list requires the permission iam.serviceAccounts.list which is present in roles such as roles/iam.serviceAccountUser named Service Account User . The service account mentioned in the error does not have one of the roles granting the permission to list service accounts or the Scopes are limiting a permission granted to the service account. Read my recommendation at the end.

Service Account Roles

Part 3

If I look at the project IAM roles, my user has the following roles:

The roles assigned to the user are not related to the roles assigned to the Compute Engine service account.

If you logged into Compute Engine using SSH and did not do anything else to authenticate then you are using Compute Engine Default Service Account credentials. Service Account and Scopes affect your permissions.

If you logged into Compute Engine using SSH and you use your own account for authentication ( gcloud auth login or similar) then your user identity is using the the permissions granted to your user account and not the Compute Engine Default service account credentials.

Part 4

The original VM is a ubuntu-16, the new VM is ubuntu-18 freshly created from a google image.

If the scopes are the same for both VMs, then your issue is the service account. Normally Compute Engine VMs use the Compute Engine Default Service Account. You can change which service account is assigned to each VM. Double check what is assigned to each VM.

Summary

I recommend that you set the scopes to Allow full access to all Cloud APIs and control the permissions via roles granted to the service account. Do not use roles such as Project Owner or Project Editor . Those roles are very powerful. Use fine grained permissions for each Google Cloud service that Compute Engine needs to access.