使用服务帐户访问GKE中的Kubernetes API

Question

I have my Kubernetes cluster running in GKE I want to run an application outside the cluster and talk to the Kubernetes API.

By using password retrieved from running:

gcloud container clusters get-credentials cluster-2 --log-http

I am able to access the API with Basic authentication. But I want to create multiple Kubernetes service accounts and configure them with required authorization and use appropriately.

So, I created service accounts and obtained the tokens using:

kubectl create serviceaccount sauser1
kubectl get serviceaccounts sauser1 -o yaml
kubectl get secret sauser1-token-<random-string-as-retrieved-from-previous-command> -o yaml

If I try to access the Kubernetes API with the obtained token using Bearer authentication then I get a 401 HTTP error. I thought that some permissions may have to be set for the service account, so based on the documentation here , I created below YAML file:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: pod-reader
subjects:
- kind: ServiceAccount
  name: sauser1
  namespace: default
roleRef:
  kind: ClusterRole
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

and tried to apply it using the below command:

kubectl apply -f default-sa-rolebinding.yaml

I got the following error:

clusterrolebinding "pod-reader" created
Error from server (Forbidden): error when creating "default-sa-rolebinding.yaml"
: clusterroles.rbac.authorization.k8s.io "pod-reader" is forbidden: attempt to g
rant extra privileges: [PolicyRule{Resources:["pods"], APIGroups:[""], Verbs:["g
et"]} PolicyRule{Resources:["pods"], APIGroups:[""], Verbs:["watch"]} PolicyRule
{Resources:["pods"], APIGroups:[""], Verbs:["list"]}] user=&{xyz@gmail.
com  [system:authenticated] map[authenticator:[GKE]]} ownerrules=[PolicyRule{Res
ources:["selfsubjectaccessreviews"], APIGroups:["authorization.k8s.io"], Verbs:[
"create"]} PolicyRule{NonResourceURLs:["/api" "/api/*" "/apis" "/apis/*" "/healt
hz" "/swagger-2.0.0.pb-v1" "/swagger.json" "/swaggerapi" "/swaggerapi/*" "/versi
on"], Verbs:["get"]}] ruleResolutionErrors=[]

I dont know how to proceed from here. Is my approach correct or am I missing something here?

UPDATE : As per the post referred by @JanosLenart in the comments, modified the kubectl command and the above error is not observed. But accessing the API, still gives 401 error. The curl command that I am using is:

curl -k -1 -H "Authorization: Bearer <token>" https://<ip-address>/api/v1/namespaces/default/pods -v

Answer 1

@Janos pointed out the potential problem, however I think you will need an actual Cloud IAM Service Account as well, because you said:

I want to run an application outside the cluster [...]

If you're authenticating to GKE from outside, I believe you can only use the Google IAM identities. (I might be wrong, if so, please let me know.)

In this case, what you need to do:

1. Create an IAM service account and download a json key file of it.
2. set GOOGLE_APPLICATION_CREDENTIALS to that file.

3. either:

   • use RBAC like in your question to give permissions to the email address of the IAM Service Account

   • use IAM Roles to give the IAM Service Account on the GKE API (eg Container Developer role is usually sufficient)

4. Use kubectl command against the cluster (make sure you have a .kube/config file with the cluster's IP/CA cert beforehand) with the environment variable above, it should work.

Answer 2

YMMV

I managed to get it work without USING an actual Cloud IAM Service Account

First, I decided to use an shell inside GKE's k8s cluster by running

kubectl run curl-random --image=radial/busyboxplus:curl -i --tty --rm

Second, I made sure I decoded my token by copying the token and then running through

pbpaste | base64 -D

Third, I created the rolebinding for the serviceaccount, NOT the username.

kubectl create clusterrolebinding shaoserverless-cluster-admin-binding --clusterrole=cluster-admin --serviceaccount=default:shaoserverless

The third step was particularly tricky but I got the inspiration since the error message used to be Unknown user \\"system:serviceaccount:default:shaoserverless\\"",

Lastly, then this works curl -k -1 -H "Authorization: Bearer <token>" https://<ip-address>/api/v1/namespaces/default/pods -v