带有工作负载身份的 Google Cloud Java SDK？

Question

Trying to figure out how to authenticate with the storage API from within a GKE cluster.

Code:

Storage storage = StorageOptions.newBuilder()
  .setCredentials(ServiceAccountCredentials.getApplicationDefault())
  .setProjectId(gcpProjectId)
  .build().getService();

getApplicationDefault() is documented to use these means to authenticate with the API:

1. Credentials file pointed to by the {@code GOOGLE_APPLICATION_CREDENTIALS} environment variable
2. Credentials provided by the Google Cloud SDK {@code gcloud auth application-default login} command
3. Google App Engine built-in credentials
4. Google Cloud Shell built-in credentials
5. Google Compute Engine built-in credentials

The application is using the GCP workload identity feature, so the application (in-cluster) service account is annotated with:

serviceAccount.annotations.iam.gke.io/gcp-service-account: my-service-account@my-project.iam.gserviceaccount.com

Now the call to the storage account fails with the following error:

{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "Primary: /namespaces/my-project.svc.id.goog with additional claims does not have storage.objects.create access to the Google Cloud Storage object.",
    "reason" : "forbidden"
  } ],
  "message" : "Primary: /namespaces/my-project.svc.id.goog with additional claims does not have storage.objects.create access to the Google Cloud Storage object."
}

This makes me think that the workload identity is not working correctly. I am expecting to receive an error message for my annotated service account and not the default one.

Is there anything else I should have been doing?

Answer 1

The answer, in part, aside from the annotation syntax, is that, just like me, you probably didn't look closely enough at this part in the documentation:

    gcloud iam service-accounts add-iam-policy-binding \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:PROJECT_ID.svc.id.goog[K8S_NAMESPACE/KSA_NAME]" \
  GSA_NAME@PROJECT_ID.iam.gserviceaccount.com

Notice the PROJECT_ID.svc.id.goog[K8S_NAMESPACE/KSA_NAME] piece. It's something they give no examples on as far as syntax but it looks like this in my terraform.

resource "google_project_iam_member" "app-binding-2" {
  role   = "roles/iam.workloadIdentityUser"
  member = "serviceAccount:${local.ws_vars["project-id"]}.svc.id.goog[mynamespace/myk8ssaname]"
}

Weirdly, I didn't know you could bind an IAM policy to a k8s service account, even more weirdly you can bind this in the terraform even if the namespace doesn't exist, much less the service account. So you can run this first before deployments.

I truly wish Google would provide better documentation and support, this took me several hours to figure out.

Answer 2

Trying to figure out how to authenticate with the storage API from within a GKE cluster.

Code:

Storage storage = StorageOptions.newBuilder()
  .setCredentials(ServiceAccountCredentials.getApplicationDefault())
  .setProjectId(gcpProjectId)
  .build().getService();

getApplicationDefault() is documented to use these means to authenticate with the API:

1. Credentials file pointed to by the {@code GOOGLE_APPLICATION_CREDENTIALS} environment variable
2. Credentials provided by the Google Cloud SDK {@code gcloud auth application-default login} command
3. Google App Engine built-in credentials
4. Google Cloud Shell built-in credentials
5. Google Compute Engine built-in credentials

The application is using the GCP workload identity feature, so the application (in-cluster) service account is annotated with:

serviceAccount.annotations.iam.gke.io/gcp-service-account: my-service-account@my-project.iam.gserviceaccount.com

Now the call to the storage account fails with the following error:

{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "Primary: /namespaces/my-project.svc.id.goog with additional claims does not have storage.objects.create access to the Google Cloud Storage object.",
    "reason" : "forbidden"
  } ],
  "message" : "Primary: /namespaces/my-project.svc.id.goog with additional claims does not have storage.objects.create access to the Google Cloud Storage object."
}

This makes me think that the workload identity is not working correctly. I am expecting to receive an error message for my annotated service account and not the default one.

Is there anything else I should have been doing?