使用 NFS 卷时，容器未在 Kubernetes 中启动

Question

i am using nfs for volume in kubernetes pod using deployment. Below are details of all files. Filename :- nfs-server.yaml

kind: Service
apiVersion: v1
metadata:
  name: nfs-service
spec:
  selector:
    role: nfs
  ports:
    # Open the ports required by the NFS server
    # Port 2049 for TCP
    - name: tcp-2049
      port: 2049
      protocol: TCP

    # Port 111 for UDP
    - name: udp-111
      port: 111
      protocol: UDP

---
kind: Pod
apiVersion: v1
metadata:
  name: nfs-server-pod
  labels:
    role: nfs
spec:
  containers:
    - name: nfs-server-container
      image: cpuguy83/nfs-server
      securityContext:
        privileged: true
      args:
        # Pass the paths to share to the Docker image
        - /exports

Both service and pod are running. Below is the output.

Now i have to use this in my web-server. Below is the details of the deployment file for web. Filename :- deployment.yaml

apiVersion: v1
kind: Service
metadata:
  name: apache-service
spec:
  selector:
    app: apache
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  type: NodePort

---
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: apache-deployment
spec:
  selector:
    matchLabels:
      app: apache
  replicas: 1 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: apache
    spec:    
      volumes:
        - name: nfs-volume
          nfs: 
            server: 10.99.56.195
            path: /exports  
      containers:
        - name: apache
          image: mobingi/ubuntu-apache2-php7:7.2
          ports:
          - containerPort: 80
          volumeMounts:
          - name: nfs-volume
            mountPath: /var/www/html 

when i am running this file without volume everything works fine. But when i am running this with nfs than pod give following error.

kubectl describe pod apache-deployment-577ffcf9bd-p8s75

Give following output:-

Name:           apache-deployment-577ffcf9bd-p8s75
Namespace:      default
Priority:       0
Node:           worker-node2/10.160.0.4
Start Time:     Tue, 09 Jul 2019 09:53:39 +0000
Labels:         app=apache
                pod-template-hash=577ffcf9bd
Annotations:    <none>
Status:         Pending
IP:             
Controlled By:  ReplicaSet/apache-deployment-577ffcf9bd
Containers:
  apache:
    Container ID:   
    Image:          mobingi/ubuntu-apache2-php7:7.2
    Image ID:       
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-p9qdb (ro)
      /var/www/html from nfs-volume (rw)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  nfs-volume:
    Type:      NFS (an NFS mount that lasts the lifetime of a pod)
    Server:    10.244.1.50
    Path:      /exports
    ReadOnly:  false
  default-token-p9qdb:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-p9qdb
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason       Age    From                   Message
  ----     ------       ----   ----                   -------
  Normal   Scheduled    6m21s  default-scheduler      Successfully assigned default/apache-deployment-577ffcf9bd-p8s75 to worker-node2
  Warning  FailedMount  4m16s  kubelet, worker-node2  MountVolume.SetUp failed for volume "nfs-volume" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume --scope -- mount -t nfs 10.244.1.50:/exports /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume
Output: Running scope as unit: run-r3a55a8a3287448a59f7e4dbefa0333af.scope
mount.nfs: Connection timed out
  Warning  FailedMount  2m10s  kubelet, worker-node2  MountVolume.SetUp failed for volume "nfs-volume" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume --scope -- mount -t nfs 10.244.1.50:/exports /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume
Output: Running scope as unit: run-r5fe7befa141d4f989e14b291afa43208.scope
mount.nfs: Connection timed out
  Warning  FailedMount  2m3s (x2 over 4m18s)  kubelet, worker-node2  Unable to mount volumes for pod "apache-deployment-577ffcf9bd-p8s75_default(29114119-5815-442a-bb97-03fa491206a4)": timeout expired waiting for volumes to attach or mount for pod "default"/"apache-deployment-577ffcf9bd-p8s75". list of unmounted volumes=[nfs-volume]. list of unattached volumes=[nfs-volume default-token-p9qdb]
  Warning  FailedMount  4s                    kubelet, worker-node2  MountVolume.SetUp failed for volume "nfs-volume" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume --scope -- mount -t nfs 10.244.1.50:/exports /var/lib/kubelet/pods/29114119-5815-442a-bb97-03fa491206a4/volumes/kubernetes.io~nfs/nfs-volume
Output: Running scope as unit: run-rd30c104228ae43df933839b6da469107.scope
mount.nfs: Connection timed out

Can Anyone please help to solve this problem.

Answer 1

1. Make sure there is no firewall between the nodes

2. Make sure nfs-utils installed on cluster nodes

3. Here is a blog posts about the docker image that you are using for nfs server, you need to do some tweaks for ports to be used by nfs server.

https://medium.com/@aronasorman/creating-an-nfs-server-within-kubernetes-e6d4d542bbb9