Terraform 和 GCP：尝试在项目级别引入模拟时出现错误 403

Question

I am quite lost when it comes to applying principles that enable service account impersonation ...

My terraform project structure has a root module per environment, base for basic infrastructure, dev for the dev environment and prod for the production environment.

terraform-infra-genesis
 ┣ base
 ┃ ┣ ...
 ┃ ┣ impersonators_x_users.tf  <- user email me@domain.com is granted iam.serviceAccounts.getAccessToken role on 'super-admin' here (On all the organization)
 ┃ ┣ ...
 ┃ ┣ providers_x_access_tokens.tf 
 ┃ ┣ service_accounts_x_roles.tf   <- 'dev-admin' service account declared here
 ┃ ┣ terraform.tfstate
 ┃ ┗ terraform.tfstate.backup
 ┣ dev <- Everything here belongs to the dev environment
 ┃ ┣ backend.tf
 ┃ ┣ data_products.tf  <- Usage of the module 'marketing-hub' here
 ┃ ┣ ...
 ┃ ┣ impersonators_x_providers_x_access_tokens.tf  <- Declaration of as_dev_admin provider to 'delegate' ressource creation (such as folders) to the dev environment "super" administrator. I also declared "as_<project>_dev_admin" that should in principle, be able to create ressources only within its own <project>
 ┣ modules
 ┃ ┣ data-products
 ┃ ┃ ┣ cmi
 ┃ ┃ ┃ ┗ feedback-hub
 ┃ ┃ ┗ ddm
 ┃ ┃ ┃ ┣ analytics-hub
 ┃ ┃ ┃ ┣ marketing-hub
 ┃ ┃ ┃ ┃ ┣ marketing_hub.tf <- Usage of module 'data-project' here to bundle all logical projects together.
 ┃ ┃ ┃ ┗ media-hub
 ┃ ┗ data-project
 ┃ ┃ ┣ data_project.tf <- Module to create a GCP project and its project-dev-admin service account here
 ┣ prod
 ┃ ┣ ...
 ┣ .gitignore
 ┗ README.md

As described in the annotations on this structure, I generally use a provider = as_dev_admin within the dev/ root-module. I successfully created dev_coupons (A GCP project) using it.

Here is the structure of my dev/impersonators_x_providers_x_access_tokens.tf

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">=3.85.0"
    }
  }
}

locals {
  tier_1_scopes = [
    "https://www.googleapis.com/auth/cloud-platform",
    "https://www.googleapis.com/auth/userinfo.email",
  ]
  tier_2_scopes = [
    "cloud-platform",
    "userinfo-email",
  ]
}
# Dev Admin impersonation
provider "google" {
  alias  = "impersonation"
  scopes = local.tier_1_scopes
}

data "google_service_account_access_token" "dev-admin" {
  provider               = google.impersonation
  target_service_account = data.terraform_remote_state.base.outputs.service-accounts.dev-admin.email
  scopes                 = local.tier_2_scopes
  lifetime               = "1200s"
}

provider "google" {
  alias        = "as_dev_admin"
  access_token = data.google_service_account_access_token.dev-admin.access_token
  region       = var.region
  zone         = var.zone
}


################################################################################
##################### Impersonation of a service account #######################
############################ as_dev_coupons_admin ##############################
################################################################################
# Copy/paste this block in order to introduce the 
# impersonation of any service account

data "google_service_account_access_token" "dev-coupons-admin" {
  provider               = google.impersonation
  target_service_account = module.marketing-hub-products.projects.coupons.admin_service_account.email
  scopes                 = local.tier_2_scopes
  lifetime               = var.lifetime
}

provider "google" {
  alias        = "as_dev_coupons_admin"
  project      = module.marketing-hub-products.projects.coupons.project_info.project_id
  access_token = data.google_service_account_access_token.dev-coupons-admin.access_token
  region       = var.region
  zone         = var.zone
}

resource "google_service_account_iam_member" "dev-coupons-admin-impersonators" {
  provider = google.as_dev_admin  # Global dev environment admin will grant this permission
  for_each = toset([
    for account in var.user_accs_impersonators_info.as_dev_coupons_admin :
    "${account.acc_type}:${account.acc_details.email}"
  ])

  service_account_id = module.marketing-hub-products.projects.coupons.admin_service_account.name
  role               = "roles/iam.serviceAccountTokenCreator"
  member             = each.value
}


################################################################################
################################### End of #####################################
############################ as_dev_coupons_admin ##############################
################################################################################

My project name is dev-coupons .

When I try to declare the additionnal provider alias as_dev_coupons_admin to a specific project dev-coupons admin, I get this error:

│ Error: googleapi: Error 403: The caller does not have permission, forbidden
│
│   with data.google_service_account_access_token.dev-coupons-admin,
│   on impersonators_x_providers_x_access_tokens.tf line 48, in data "google_service_account_access_token" "dev-coupons-admin":
│   48: data "google_service_account_access_token" "dev-coupons-admin" {
│

I don't understand why creating the "google_service_account_access_token" "dev_coupons_admin" returns a 403 ... At first, I thought it is because some parent module's provider was interfering, but no, here we are at the base module dev , with the same credentials that created the whole dev environment ressources, with the same associated user email, yet this denial of access is returned.

I then enabled logs export TF_VARS=DEBUG; export TF_LOG_PATH="terraform_log.txt" export TF_VARS=DEBUG; export TF_LOG_PATH="terraform_log.txt" , and I find this line:

---[ REQUEST ]---------------------------------------
POST /v1/projects/-/serviceAccounts/dev-coupons-admin@<redacted_project_id>.iam.gserviceaccount.com:generateAccessToken?alt=json&prettyPrint=false HTTP/1.1
Host: iamcredentials.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/1.2.9 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/dev
Content-Length: 129
Content-Type: application/json
X-Goog-Api-Client: gl-go/1.18.1 gdcl/0.92.0
Accept-Encoding: gzip

{
 "lifetime": "1200s",
 "scope": [
  "https://www.googleapis.com/auth/cloud-platform",
  "https://www.googleapis.com/auth/userinfo.email"
 ]
}

-----------------------------------------------------: timestamp=2022-09-21T21:25:58.442+0200
2022-09-21T21:25:58.534+0200 [INFO]  provider.terraform-provider-google_v4.36.0_x5: 2022/09/21 21:25:58 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 403 Forbidden
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Wed, 21 Sep 2022 19:26:04 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "error": {
    "code": 403,
    "message": "The caller does not have permission",
    "errors": [
      {
        "message": "The caller does not have permission",
        "domain": "global",
        "reason": "forbidden"
      }
    ],
    "status": "PERMISSION_DENIED"
  }
}

Perhaps it is trying to access "all" projects via the designated service account? See - in /v1/projects/-/serviceAccounts/ ?

If you are able to shed some light on where my understanding is lacking, I would greatly appreciate it.

EDIT: dev-coupons-admin is the owner of dev-coupons

Answer 1

TL;DR me@domain.com has the right to create tokens using a dev environment super admin. This does not mean that right cascades to projects within this same environment, you have to redeclare the account as token creator associated with the right service account.

I think I figured out this problem. The key is to understand the inheritance of roles for service account token_creators .

• Within my organization the as_dev_admin is used to create resources as the default provider.
  • There is a service account that has the right for creating the infrastructure elements.
  • The user email me@domain.com that has the roles/iam.serviceAccountTokenCreator is associated to this service account via a project within the base infrastructre, higher in the hierarchy, but not a direct parent.
  • -> It is apparently erroneous to assume the user associated to service account will have the same right within child elements of the infrastructure.

This means that this role for the impersonator is not inherited within the data-project created in dev environment. Once you try to impersonate the dev-coupons-admin@<redacted_project_id>.iam.gserviceaccount.com using me@domain.com , it will not transfer to this email account because it was present in dev-admin@<some_common_project_id>.iam.gserviceaccount.com , the dev super admin.