如何从 Gitlab 和 Helm 部署到 minikube

Question

我正在尝试使用 gitlab-ci 管道在我的本地 minikube 上部署 java spring 项目..但我不断得到

ERROR: Job failed (system failure): prepare environment: setting up credentials: secrets is forbidden: User "system:serviceaccount:maverick:default" cannot create resource "secrets" in API group "" in the namespace "maverick". Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

我已经在“maverick”命名空间上安装了 gitlab-runner

apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab-runner
  namespace: maverick
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: gitlab-runner
  namespace: maverick
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["list", "get", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get"]
  - apiGroups: [""]
    resources: ["pods/attach"]
    verbs: ["list", "get", "create", "delete", "update"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["list", "get", "create", "delete", "update"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["list", "get", "watch", "create", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: gitlab-runner
  namespace: maverick
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: maverick
subjects:
- namespace: maverick
  kind: ServiceAccount
  name: gitlab-runner

和价值观

gitlabUrl: https://gitlab.com/
runnerRegistrationToken: ".... my token .... "

runners:
  privileged: false
  tags: k8s
  serviceAccountName: gitlab-runner

我的 gitlab-ci.yml 是这样的：

docker-build-job:
  stage: docker-build
  image: $MAVEN_IMAGE
  script:
    - mvn jib:build -Djib.to.image=${CI_REGISTRY_IMAGE}:latest -Djib.to.auth.username=${CI_REGISTRY_USER} -Djib.to.auth.password=${CI_REGISTRY_PASSWORD}

deploy-job:
  image: alpine/helm:3.2.1
  stage: deploy
  tags:
    - k8s
  script:
    - helm upgrade ${APP_NAME} ./charts --install --values=./charts/values.yaml --namespace ${APP_NAME}
  rules:
    - if: $CI_COMMIT_BRANCH == 'master'
      when: always

并且图表文件夹有 deployment.yaml 像这样：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: maverick
  namespace: maverick
spec:
  replicas: 1
  selector:
    matchLabels:
      app: maverick
  template:
    metadata:
      labels:
        app: maverick
    spec:
      containers:
        - name: maverick
          image: registry.gitlab.com/gfalco77/maverick:latest
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8001
      imagePullSecrets:
        - name: registry-credentials
---
apiVersion: v1
kind: Service
metadata:
  name: maverick
spec:
  ports:
    - name: maverick
      port: 8001
      targetPort: 8001
      protocol: TCP
  selector:
    app: maverick

还有一个我根据https://chris-vermeulen.com/using-gitlab-registry-with-kubernetes/创建的注册表凭据，它们安装在特立独行的命名空间中

apiVersion: v1
kind: Secret
metadata:
  name: registry-credentials
  namespace: maverick
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: .. base64 creds ..

我可以看到 gitlab-runner 对 apigroup "" 具有创建权限。但它似乎仍然无法从注册表下载图像，不知道出了什么问题？

提前致谢

Answer 1

问题解决了添加以下 ClusterRole 和 ClusterRoleBinding，尤其是第二个名称为“default”的问题之后 gitlab 中的作业继续，然后尝试使用用户system:serviceaccount:maverick:gitlab-runner ，但它在我需要的其他东西上失败弄清楚

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: cluster-admin
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["list", "get", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get"]
  - apiGroups: [""]
    resources: ["pods/attach"]
    verbs: ["list", "get", "create", "delete", "update"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["list", "watch", "get", "create", "delete", "update"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["list", "get", "watch", "create", "delete", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-admin-role
subjects:
  - kind: ServiceAccount
    name: gitlab-runner
    namespace: maverick
roleRef: # referring to your ClusterRole
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-admin-role
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: default
    namespace: maverick