How to use OpenStack Cinder to create storage class and dynamically provision persistent volume in Kubernetes Cluster

Question

Recently when practicing kubernetes , I found there is no doc and example specifically explaining how to use cinder correctly in kubernetes.

So how to setup cinder to be used in kubernetes ?

Answer 1

I did some experiment and worked out how to setup cinder with kubernetes. Just find a suitable to document and share.

Preparation

• kubernetes cluster
• openstack environment and make sure cinder service is available

Background

From my investigation, component kube-controller-manager is responsible for loading volume plugins and related in Kubernetes. So we could make cinder available by adjusting kube-controller-manager configuration.

Steps

1. Prepare cloud.conf file to contain your openstack creds

Prepare your openstack creds and saved as a file , for example /etc/kubernetes/cloud.conf in kubernetes control panel which kube-controller-manager locates. The following is example for cloud.conf

[Global]
auth-url=$your_openstack_auth_url
username=$your_openstack_user
password=$your_user_pw
region=$your_openstack_reigon
tenant-name=$your_project_name
domain-name=$your_domain_name
ca-file=$your_openstack_ca

Most could be found from your stackrc file. And ca-file item is optional, depending on if your openstack auth url is http or https

2. Adjust kube-controller-manager start configuration

This link is a full detail options for kube-controller-manager ( https://kubernetes.io/docs/admin/kube-controller-manager/ )

Actually we should add two extra parameters based on your current one

--cloud-provider=openstack
--cloud-config=/etc/kubernetes/cloud.conf

There are mainly two ways to start kube-controller-manager : 1) using systemd 2) using static pod .

Just one tips, if you are using static pod for kube-controller-manager , make sure you have mount all files such as cloud.conf or openstack ca file into your container.

Verification

We will create a storageclass, and use this storageclass to create persistent volume dynamically.

1. Create a storageclass named standard :

demo-sc.yml:

apiVersion: storage.k8s.io/v1beta1
kind: StorageClass
metadata:
  name: standard
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
  labels:
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: EnsureExists
provisioner: kubernetes.io/cinder

Using command kubectl create -f demo-sc.yml to create and using command kubectl get sc to verify if it created correctly

NAME                 TYPE
standard (default)   kubernetes.io/cinder 

2. Create a PersistentVolumeClaim to use StorageClass provison a Persistent Volume in Cinder:

demo-pvc.yml:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: cinder-claim
  annotations:
    volume.beta.kubernetes.io/storage-class: "standard"
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Creating PVC by kubectl create -f demo-pvc.yml

And now checking by command kubectl get pvc

NAME           STATUS    VOLUME                                         CAPACITY   ACCESSMODES   STORAGECLASS   AGE
cinder-claim   Bound     pvc-5dd3d62e-9204-11e7-bc43-fa163e0e0379   1Gi          RWO           standard       23h

And in openstack environment, checking by command cinder list | grep pvc-5dd3d62e-9204-11e7-bc43-fa163e0e0379cinder list | grep pvc-5dd3d62e-9204-11e7-bc43-fa163e0e0379

    root@ds0114:~# cinder list | grep pvc-5dd3d62e-9204-11e7-bc43- fa163e0e0379
| ddd8066d-2e16-4cb2-a89e-cd9d5b99ef1b | available | kubernetes-dynamic- pvc-5dd3d62e-9204-11e7-bc43-fa163e0e0379 |  1   |   CEPH_SSD  |  false   |                                       |

So now StorageClass is working well using Cinder in Kubernetes.

Answer 2

Thanks a lot for your great share!
The solution works for me ( K8S 1.14.3 , OpenStack Queen ), and I just added snippets of parameter/volumeMounts/volume like below:

Parameter :

- --cloud-provider=openstack  
- --cloud-config=/etc/kubernetes/cloud-config  

volumeMounts :

-- mountPath: /etc/kubernetes/cloud-config  
   name: cloud  
   readOnly: true  

volume :

-- hostPath:  
     path: /etc/kubernetes/cloud.conf  
     type: FileOrCreate  
   name: cloud