Error while creating persistanceVolume with mysql on kubernetes

Question

I'm trying to create a kubernetes persistence volume to connect it with a spring boot application, but I found this error on a pod of the persistance volume: mysqld: Table 'mysql.plugin' doesn't exist . A secret file is created for both user and admin and a configMap file also to map the spring boot image to the service of mysql. Here is my deployment file:

# Define a 'Service' To Expose mysql to Other Services
apiVersion: v1
kind: Service
metadata:
  name: mysql  # DNS name
  labels:
    app: mysql
    tier: database
spec:
  ports:
    - port: 3306
      targetPort: 3306
  selector:       # mysql Pod Should contain same labels
    app: mysql
    tier: database
  clusterIP: None  # We Use DNS, Thus ClusterIP is not relevant
---
# Define a 'Persistent Volume Claim'(PVC) for Mysql Storage, dynamically provisioned by cluster
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim # name of PVC essential for identifying the storage data
  labels:
    app: mysql
    tier: database
spec:
  accessModes:
    - ReadWriteOnce   #This specifies the mode of the claim that we are trying to create.
  resources:
    requests:
      storage: 1Gi    #This will tell kubernetes about the amount of space we are trying to claim.
---

apiVersion: v1
kind: PersistentVolume
metadata:
  name: task-pv-volume
  labels:
    type: local
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"
---

# Configure 'Deployment' of mysql server
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
  labels:
    app: mysql
    tier: database
spec:
  selector: # mysql Pod Should contain same labels
    matchLabels:
      app: mysql
      tier: database
  strategy:
    type: Recreate
  template:
    metadata:
      labels: # Must match 'Service' and 'Deployment' selectors
        app: mysql
        tier: database
    spec:
      containers:
        - image: mysql:latest # image from docker-hub
          args:
            - "--ignore-db-dir"
            - "lost+found" # Workaround for https://github.com/docker-library/mysql/issues/186
          name: mysql
          env:
            - name: MYSQL_ROOT_PASSWORD # Setting Root Password of mysql From a 'Secret'
              valueFrom:
                secretKeyRef:
                  name: db-admin # Name of the 'Secret'
                  key: password   # 'key' inside the Secret which contains required 'value'
            - name: MYSQL_USER # Setting USER username on mysql From a 'Secret'
              valueFrom:
                secretKeyRef:
                  name: db-user
                  key: username
            - name: MYSQL_PASSWORD # Setting USER Password on mysql From a 'Secret'
              valueFrom:
                secretKeyRef:
                  name: db-user
                  key: password
            - name: MYSQL_DATABASE # Setting Database Name from a 'ConfigMap'
              valueFrom:
                configMapKeyRef:
                  name: db-config
                  key: name
          ports:
            - containerPort: 3306
              name: mysql
          volumeMounts:        # Mounting volume obtained from Persistent Volume Claim
            - name: mysql-persistent-storage
              mountPath: /var/lib/mysql #This is the path in the container on which the mounting will take place.
      volumes:
        - name: mysql-persistent-storage # Obtaining 'volume' from PVC
          persistentVolumeClaim:
            claimName: mysql-pv-claim 

I'm using the latest image of mysql with docker. Is there any configuration in my file that I must change ?

Answer 1

First of all, using lastest mysql container you are using mysql:8.

--ignore-db-dir flag does not exist in version 8 so you should see in logs this error:

2020-09-21 0 [ERROR] [MY-000068] [Server] unknown option '--ignore-db-dir'.
2020-09-21 0 [ERROR] [MY-010119] [Server] Aborting

After removing this flag, mysql stops crashing, but there appears to be other problem.

The hostPath volume does not work as expected. When I runned:

$ kubectl get pvc
NAME             STATUS   VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
mysql-pv-claim   Bound    pvc-xxxx   1Gi        RWO            standard       4m16s

you can see that STORAGECLASS is set to standard, and its causing k8s to provision a pv, instead of using already created one.

# kubectl get pv
NAME            CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                    STORAGECLASS   REASON   AGE
pvc-xxxx        1Gi        RWO            Delete           Bound       default/mysql-pv-claim   standard                6m31s
task-pv-volume  1Gi        RWO            Retain           Available                                                    6m31s

As you probably see the task-pv-volume status is Available and that means that nothing is using it.

To use it you you need to set storageClassName for Persistent Volume Claim to empty string: "" and maybe add volumeName like following:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
  labels:
    app: mysql
    tier: database
spec:
  storageClassName: ""
  volumeName: task-pv-volume
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

After doing this hostPath volume will work as expected, but there is still a problem with lost+found directory.

My solution would be to create another directory in /mnt/data/ dir eg /mnt/data/mysqldata/ and use /mnt/data/mysqldata/ as hostPath in PersistentVolume object.

Other solution would be using older version of mysql that supports --ignore-db-dir flag.