Unable to create a RabbitMQ instance using RabbitMQ cluster Kubernetes operator

Question

I'm trying to create a RabbitMQ instance using RabbitMQ cluster Kube.netes operator, but there is an issue with PersistentVolumeClaims. I'm running Kube.netes 1.18.8 using Docker Desktop for Windows.

I have installed the operator like this:

kubectl apply -f "https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml"

I have created this very simple configuration for the instance according to the documentation:

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: nccrabbitmqcluster

It seems to create all of the objects it is supposed to create, but the pod gets stuck on pending state:

$ kubectl get all | grep rabbit
pod/nccrabbitmqcluster-server-0         0/1     Pending   0          14m
service/nccrabbitmqcluster                  ClusterIP      10.100.186.115   <none>        5672/TCP,15672/TCP   14m
service/nccrabbitmqcluster-nodes            ClusterIP      None             <none>        4369/TCP,25672/TCP   14m
statefulset.apps/nccrabbitmqcluster-server     0/1     14m

There seems to be an unbound PVC according to the pod's events:

$ kubectl describe pod/nccrabbitmqcluster-server-0 | tail -n 5
Events:
  Type     Reason            Age        From               Message
  ----     ------            ----       ----               -------
  Warning  FailedScheduling  <unknown>  default-scheduler  running "VolumeBinding" filter plugin for pod "nccrabbitmqcluster-server-0": pod has unbound immediate PersistentVolumeClaims
  Warning  FailedScheduling  <unknown>  default-scheduler  running "VolumeBinding" filter plugin for pod "nccrabbitmqcluster-server-0": pod has unbound immediate PersistentVolumeClaims

According to the events of the PVC, it is waiting for a volume to be created:

$ kubectl describe pvc persistence-nccrabbitmqcluster-server-0
Name:          persistence-nccrabbitmqcluster-server-0
Namespace:     default
StorageClass:  hostpath
Status:        Pending
Volume:
Labels:        app.kubernetes.io/component=rabbitmq
               app.kubernetes.io/name=nccrabbitmqcluster
               app.kubernetes.io/part-of=rabbitmq
Annotations:   volume.beta.kubernetes.io/storage-provisioner: docker.io/hostpath
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Mounted By:    nccrabbitmqcluster-server-0
Events:
  Type    Reason                Age                 From                         Message
  ----    ------                ----                ----                         -------
  Normal  ExternalProvisioning  27s (x23 over 19m)  persistentvolume-controller  waiting for a volume to be created, either by external provisioner "docker.io/hostpath" or manually created by system administrator

My understanding is that docker.io/hostpath is the correct provisioner:

$ kubectl get storageclasses
NAME                 PROVISIONER          RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
hostpath (default)   docker.io/hostpath   Delete          Immediate           false                  20d

I can't see any PVs related to PCS:

$ kubectl get pv | grep rabbit

Why isn't the volume created automatically and what should I do?

Answer 1

Yes, your local hostpath can not work as dynamic volume provisioner. This operator needs an storageclassname which can dynamically create PVs. In your case, your operator waiting continuously for PV to get created. In stead you can manually create an PV and PVC if you are doing in local machine. Check this example - https://github.com/rabbitmq/cluster-operator/blob/main/docs/examples/multiple-disks/rabbitmq.yaml

If you are going to try any cloud provider like AWS then its pretty easy. Deploy EBS CSI driver in your cluster which will create an storageclass for you and that storageclass will provision dynamic volumes.