Kubernetes MySQL pod stuck with CrashLoopBackOff

Question

I'm trying to follow this guide to set up a MySQL instance to connect to. The Kubernetes cluster is run on Minikube.

From the guide, I have this to set up my persistent volume:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: mysql-pv-volume
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

I ran kubectl describe pods/mysql-c85f7f79c-kqjgp and got this:

Start Time:   Wed, 29 Jan 2020 09:09:18 -0800
Labels:       app=mysql
              pod-template-hash=c85f7f79c
Annotations:  <none>
Status:       Running
IP:           172.17.0.13
IPs:
  IP:           172.17.0.13
Controlled By:  ReplicaSet/mysql-c85f7f79c
Containers:
  mysql:
    Container ID:   docker://f583dad6d2d689365171a72a423699860854e7e065090bc7488ade2c293087d3
    Image:          mysql:5.6
    Image ID:       docker-pullable://mysql@sha256:9527bae58991a173ad7d41c8309887a69cb8bd178234386acb28b51169d0b30e
    Port:           3306/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    137
      Started:      Wed, 29 Jan 2020 19:40:21 -0800
      Finished:     Wed, 29 Jan 2020 19:40:22 -0800
    Ready:          False
    Restart Count:  7
    Environment:
      MYSQL_ROOT_PASSWORD:  password
    Mounts:
      /var/lib/mysql from mysql-persistent-storage (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-5qchv (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  mysql-persistent-storage:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  mysql-pv-claim
    ReadOnly:   false
  default-token-5qchv:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-5qchv
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  <unknown>             default-scheduler  Successfully assigned default/mysql-c85f7f79c-kqjgp to minikube
  Normal   Pulled     10h (x5 over 10h)     kubelet, minikube  Container image "mysql:5.6" already present on machine
  Normal   Created    10h (x5 over 10h)     kubelet, minikube  Created container mysql
  Normal   Started    10h (x5 over 10h)     kubelet, minikube  Started container mysql
  Warning  BackOff    2m15s (x50 over 10h)  kubelet, minikube  Back-off restarting failed container

When I get the logs via kubectl logs pods/mysql-c85f7f79c-kqjgp , I only see this:

2020-01-30 03:50:47+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.6.47-1debian9 started.
2020-01-30 03:50:47+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2020-01-30 03:50:47+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.6.47-1debian9 started.

Is there a better way to debug? Why are the logs empty?

Answer 1

Hmm really odd, I changed my mysql-deployment.yml to use MySQL 5.7 and it seems to have worked...

  - image: mysql:5.7

Gonna take this as the solution until further notice/commentary.

Answer 2

I've faced the same issue and I solved it by increasing mysql container resources from 128Mi to 512Mi. Following configuration works for me.

containers:
  - name: cant-drupal-mysql
    image: mysql:5.7
    resources:
      limits:
        memory: "512Mi"
        cpu: "1500m"