兩個 Kubernetes 集群對 RBAC 的作用不同

Question

我創建了一個需要訪問列表、創建、更新和刪除不同 Kubernetes 資源的應用程序，我為它創建了一個集群角色，如下所示。 在我在 Microk8s 上運行的本地 K8s 集群上一切正常，但是當我將它部署在具有相同 K8s 版本的裸機集群上時，我收到了我沒有正確訪問權限的錯誤。

這是怎么可能的（兩者都應該表現相同），有沒有辦法提前找到這些錯誤？

我的集群角色：

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: {{ .Release.Namespace }}-cluster-manager-role
rules:
- apiGroups: ["","apps","core", "autoscaling"] # --> I was getting error that I cannot create HPA but after I added "autoscaling" to the apigroup now I can create HPA
  resources: ["*", "namespaces"]
  verbs: ["get", "watch", "list", "patch", "create", "delete", "update"]

# ================
# Current clusterrole on microk8s (which allows me to do all the things)
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: "2021-05-31T12:05:58Z"
  name: default-cluster-manager-role
  resourceVersion: "937643"
  selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/default-cluster-manager-role
  uid: 16fb63d6-1261-48a9-bc7f-5c8fffb72c9d
rules:
- apiGroups:
  - ""
  - apps
  - core
  resources:
  - '*'
  - namespaces
  verbs:
  - get
  - watch
  - list
  - patch
  - create
  - delete
  - update

Kubernetes 版本：

# Microk8s
$ kubectl version 
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:11:31Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.15", GitCommit:"2adc8d7091e89b6e3ca8d048140618ec89b39369", GitTreeState:"clean", BuildDate:"2020-09-02T11:31:21Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

# Bare-metal
$ kubectl version 
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.3", GitCommit:"b3cbbae08ec52a7fc73d334838e18d17e8512749", GitTreeState:"clean", BuildDate:"2019-11-13T11:23:11Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.15", GitCommit:"2adc8d7091e89b6e3ca8d048140618ec89b39369", GitTreeState:"clean", BuildDate:"2020-09-02T11:31:21Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

我得到的一些錯誤：

time="2021-06-22T08:45:31Z" level=error msg="Getting list of PVCs for namespace wws-test failed." func=src/k8s.CreateClusterRole file="/src/k8s/k8s.go:1304"
time="2021-06-22T08:45:31Z" level=error msg="clusterroles.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:wws:wws-cluster-manager-sa\" cannot create resource \"clusterroles\" in API group \"rbac.authorization.k8s.io\" at the cluster scope" func=src/k8s.CreateClusterRole file="/src/k8s/k8s.go:1305"
time="2021-06-22T08:45:31Z" level=error msg="Getting list of PVCs for namespace wws-test failed." func=src/k8s.CreateClusterRoleBinding file="/src/k8s/k8s.go:1232"
time="2021-06-22T08:45:31Z" level=error msg="clusterrolebindings.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:wws:wws-cluster-manager-sa\" cannot create resource \"clusterrolebindings\" in API group \"rbac.authorization.k8s.io\" at the cluster scope" func=src/k8s.CreateClusterRoleBinding file="/src/k8s/k8s.go:1233"
time="2021-06-22T08:45:32Z" level=error msg="Getting list of PVCs for namespace wws-test failed." func=src/k8s.CreateRole file="/src/k8s/k8s.go:1448"
time="2021-06-22T08:45:32Z" level=error msg="roles.rbac.authorization.k8s.io is forbidden: User \"system:serviceaccount:wws:wws-cluster-manager-sa\" cannot create resource \"roles\" in API group \"rbac.authorization.k8s.io\" in the namespace \"wws-test\"" func=src/k8s.CreateRole file="/src/k8s/k8s.go:1449"

Answer 1

您應該查看應用於 ServiceAccount 的 ClusterRoleBindings (k get ClusterRoleBinding -o wide)：system:serviceaccount:wws:wws-cluster-manager-sa)

我猜在 Minikube 上，您的用戶可以在本地集群上做任何事情。 但是，真正的集群不允許您使用默認用戶創建新的 ClusterRoles/CluterRoleBindings。

Answer 2

我不知道為什么會這樣，但我通過對apiGroups 、 resources和verbs所有三個字段使用*解決了這個問題：

rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]

我知道這不是一個干凈和完美的解決方案，特別是如果您想對角色應該有權訪問的角色和資源或動詞進行更多控制，但因為沒有人（即使我將其發布在 Kubernetes repo github 上作為問題） ) 知道為什么會發生這種情況，我沒有時間深入研究這個我接受我自己的答案。