Jenkins容器在Kubernetes集群上的持久性-PersistentVolumeClaim（VMware / Vsphere）

Question

Trying to persist my jenkins jobs on to vsphere storage when I delete the deployments/services.

I've tried using the standard approach: used StorageClass, then made a PersistentVolumeClaim which is referenced in the .ayml file that will create the deployments.

storage-class.yml:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: mystorage
provisioner: kubernetes.io/vsphere-volume
parameters:
  diskformat: zeroedthick

persistent-volume-claim.yml:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc0003
spec:
  storageClassName: mystorage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 15Gi

jenkins.yml:

---
apiVersion: v1
kind: Service
metadata:
  name: jenkins-auto-ci
  labels:
    app: jenkins-auto-ci
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: jenkins-auto-ci
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins-auto-ci
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jenkins-auto-ci
    spec:
      containers:
      - name: jenkins-auto-ci
        image: jenkins
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        env:
        - name: GET_HOSTS_FROM
          value: dns
        ports:
        - name: http-port
          containerPort: 80
        - name: jnlp-port
          containerPort: 50000
        volumeMounts:
        - name: jenkins-home
          mountPath: "/var"
      volumes:
        - name: jenkins-home
          persistentVolumeClaim:
            claimName: pvc0003

I expect the jenkins jobs to persist when I delete and recreate the deployments.

Answer 1

You should create VMDK which is Virtual Machine Disk.

You can do that using govc which is vSphere CLI.

govc datastore.disk.create -ds datastore1 -size 2G volumes/myDisk.vmdk

Or using ESXi CLI by ssh into the host as root and executing:

vmkfstools -c 2G /vmfs/volumes/datastore1/volumes/myDisk.vmdk

Once this is done you should create your PV let's call it vsphere_pv.yaml which might look like the following:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv0001
spec:
  capacity:
    storage: 2Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  vsphereVolume:
    volumePath: "[datastore1] volumes/myDisk"
    fsType: ext4

The datastore1 in this example was created in root folder of vCenter, if you have it in a different location you need to change the volumePath . If it's located in DatastoreCluster then set volumePath to "[DatastoreCluster/datastore1] volumes/myDisk" .

Apply the yaml to the Kubernetes by kubectl apply -f vsphere_pv.yaml

You can check if it was created by describing it kubectl describe pv pv0001

Now you need PVC let's call it vsphere_pvc.yaml to consume PV.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc0001
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

Apply the yaml to the Kubernetes by kubectl apply -f vsphere_pvc.yaml

You can check if it was created by describing it kubectl describe pvc pv0001

Once this is done your yaml might be looking like the following:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins-auto-ci
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jenkins-auto-ci
    spec:
      containers:
      - name: jenkins-auto-ci
        image: jenkins
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        env:
        - name: GET_HOSTS_FROM
          value: dns
        ports:
        - name: http-port
          containerPort: 80
        - name: jnlp-port
          containerPort: 50000
        volumeMounts:
        - name: jenkins-home
          mountPath: "/var"
      volumes:
        - name: jenkins-home
          persistentVolumeClaim:
            claimName: pvc0001

All this is nicely explained on Vmware GitHub vsphere-storage-for-kubernetes .