GKE / Cloud IAM 工作负载身份设置错误 403

Question

We have setup a CloudSQL proxy as a sidecar container for one of our Java / Tomcat based app.

Here's how we setup workload identity to enable our app to connect to CloudSQL through cloudsql proxy:

1. Created Cloud IAM Service Account and gave it SQL Client permission:

2. Setup Policy Binding as follows:

    gcloud iam service-accounts add-iam-policy-binding \\ --role roles/iam.workloadIdentityUser \\ --member "serviceAccount:[PROJECT_ID].svc.id.goog[default/default]" \\ [GSA_NAME]@[PROJECT_ID].iam.gserviceaccount.com

3. Added annotation to GKE Service Account:

    kubectl annotate serviceaccount \\ --namespace [K8S_NAMESPACE] \\ [KSA_NAME] \\ iam.gke.io/gcp-service-account=[GSA_NAME]@[PROJECT_ID].iam.gserviceaccount.com

4. But when we test this using:

    kubectl run --rm -it \\ --generator=run-pod/v1 \\ --image google/cloud-sdk:slim \\ --serviceaccount [KSA_NAME] \\ --namespace [K8S_NAMESPACE] \\ workload-identity-test

Despite doing everything correctly as explained on this page still results in:

Error 403: The client is not authorized to make this request., notAuthorized

Answer 1

It turns out there is a glitch in Google Cloud IAM which seems to affect service accounts.

1. Delete the existing Cloud IAM service account created in step 1 and role associated with it.
2. Recreate the service account again (with exam same permissions)

which should fix the issue. Of course you'll have to redo step 2 and 3 again to complete the setup but it works.