Vault 服务帐户是否应使用默认服务帐户 API 令牌向 Kube.netes 进行身份验证？

Question

I've been followingthis tutorial to set up vault and kube.netes on minikube with helm.

It seems to me the vault service account is using the default service account JWT token to access the API to authenticate. Should the vault service account instead be using its own token, not the default one?

Here are the relevant configuration steps:

# Configure the auth.
vault write auth/kubernetes/config \
      token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
      kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443" \
      kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
      disable_iss_validation=true

# Configure the policy.
vault policy write webapp - <<EOF
path "secret/data/webapp/config" {
  capabilities = ["read"]
}
EOF

# Create the role.
vault write auth/kubernetes/role/webapp \
      bound_service_account_names=vault \
      bound_service_account_namespaces=default \
      policies=webapp \
      ttl=24h

The client (expressjs) on the webapp pod reads the mounted token to authenticate:

...
await axiosInst.post("/auth/kubernetes/login",
  {jwt: fs.readFileSync(process.env.JWT_PATH, {encoding: "utf-8"}),
  role: "webapp"});
...

However, the token at /var/run/secrets/kube.netes.io/serviceaccount/token on the webapp pod is the default service account token, not the vault service account token right?

Am I doing something wrong here?

web-deploy.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-pod
  template:
    metadata:
      labels:
        app: web-pod
    spec:
      serviceAccountName: vault
      containers:
      - name: web
        image: kahunacohen/hello-k8s
        imagePullPolicy: IfNotPresent
        env:
          - name: VAULT_ADDR
            value: 'http://vault:8200'
          - name: JWT_PATH
            value: '/var/run/secrets/kubernetes.io/serviceaccount/token'
          - name: SERVICE_PORT
            value: '8080'
        envFrom:
          - configMapRef:
              name: web-configmap
        ports:
        - containerPort: 3000
          protocol: TCP

I describe the vault sa:

$ kubectl describe  serviceaccounts vault
Name:                vault
Namespace:           default
Labels:              app.kubernetes.io/instance=vault
                     app.kubernetes.io/managed-by=Helm
                     app.kubernetes.io/name=vault
                     helm.sh/chart=vault-0.14.0
Annotations:         meta.helm.sh/release-name: vault
                     meta.helm.sh/release-namespace: default
Image pull secrets:  <none>
Mountable secrets:   vault-token-jhp5q
Tokens:              vault-token-jhp5q
Events:              <none>

I describe the secret associated with this sa:

$ kubectl describe secret vault-token-jhp5q
Name:         vault-token-jhp5q
Namespace:    default
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: vault
              kubernetes.io/service-account.uid: 9a091a35-9292-4d77-abac-a7d62f312d27

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1111 bytes
namespace:  7 bytes
token:      eyJhb...

But when I cat the token on my pod it doesn't match above:

$ kubectl exec web-deployment-66968775cb-8td6c -- cat  /var/run/secrets/kubernetes.io/serviceaccount/token
...

pod yaml:

$ kubectl get pods web-deployment-66968775cb-8td6c -o yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2021-08-14T19:32:54Z"
  generateName: web-deployment-66968775cb-
  labels:
    app: web-pod
    app.kubernetes.io/managed-by: skaffold
    pod-template-hash: 66968775cb
    skaffold.dev/run-id: f58f771b-258a-4304-baf6-1cc4d9bc3029
  name: web-deployment-66968775cb-8td6c
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: web-deployment-66968775cb
    uid: 6e62ba08-8b56-402a-ae10-e5d95f93dc39
  resourceVersion: "1444983"
  uid: 991199f5-0e53-45b9-a4d7-ab1d3cc44079
spec:
  containers:
  - env:
    - name: VAULT_ADDR
      value: http://vault:8200
    - name: JWT_PATH
      value: /var/run/secrets/kubernetes.io/serviceaccount/token
    - name: SERVICE_PORT
      value: "8080"
    envFrom:
    - configMapRef:
        name: web-configmap
    image: kahunacohen/hello-k8s:fb961e142e46220f11853eba6f50eaf94510a95dc08046ad8a1517d92fcb2d85
    imagePullPolicy: IfNotPresent
    name: web
    ports:
    - containerPort: 3000
      protocol: TCP
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-zjkvp
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: minikube
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: vault
  serviceAccountName: vault
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: kube-api-access-zjkvp
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          expirationSeconds: 3607
          path: token
      - configMap:
          items:
          - key: ca.crt
            path: ca.crt
          name: kube-root-ca.crt
      - downwardAPI:
          items:
          - fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
            path: namespace
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2021-08-14T19:32:54Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2021-08-15T15:01:02Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2021-08-15T15:01:02Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2021-08-14T19:32:54Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: docker://4759183417a8ef0bbee8f28bf345aed373371f7ca462b9a5f5a966eba33117d2
    image: kahunacohen/hello-k8s:b65b899-dirty
    imageID: docker://sha256:fb961e142e46220f11853eba6f50eaf94510a95dc08046ad8a1517d92fcb2d85
    lastState:
      terminated:
        containerID: docker://87636058668d022160149f176cd3c565bf17c239e97c55d873e42c9c074215d6
        exitCode: 255
        finishedAt: "2021-08-15T15:00:01Z"
        reason: Error
        startedAt: "2021-08-14T19:32:55Z"
    name: web
    ready: true
    restartCount: 1
    started: true
    state:
      running:
        startedAt: "2021-08-15T15:01:01Z"
  hostIP: 192.168.49.2
  phase: Running
  podIP: 172.17.0.10
  podIPs:
  - ip: 172.17.0.10
  qosClass: BestEffort
  startTime: "2021-08-14T19:32:54Z"

Answer 1

"/var/run/secrets/kube.netes.io/serviceaccount/token" on the webapp pod is the default service account token, right?

No, it depends on the Pod YAML.

1. Case 1 : If you didn't set the service account for your pod, it uses the default service account.
2. Case 2 : If you set the service account for your pod, it will use the jwt token of that service account. In your case, it's vault .

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  serviceAccountName: vault
... ... ...

• ServiceAccount Admission Controller

Kube.netes Auth & Vault:

• You need a service account (say admin ) that will be used to configure (ie. token_reviewer_jwt=... ) Vault k8s authentication. It will have system-authdelegator RBAC role assigned to it. Now it has the power to validate other service account JWT tokens.
• You have an application running with another service account (say client ), the vault admin has created a k8s auth role (ie. bound_service_account_names=client ) with it.
• Your App will send the client's JWT to Vault--> Vault will send that to kube-api, with both admin's JWT (to introduce kube-api that he is the admin) and client's JWT ( whom the admin want to verify).