Accessing Postgresql data of Kubernetes cluster

Question

I have kubernetes cluster with two replicas of a PostgreSQL database in it, and I wanted to see the values stored in the database.

When I exec myself into one of the two postgres pod ( kubectl exec --stdin --tty [postgres_pod] -- /bin/bash ) and check the database from within, I have only a partial part of the DB. The rest of the DB data is on the other Postgres pod, and I don't see any directory created by the persistent volumes with all the database stored.

So in short I create 4 tables; in one postgres pod I have 4 tables but 2 are empty, in the other postgres pod there are 3 tables and the tables that were empty in the first pod, here are filled with data.

Why the pods don't have the same data in it?

How can I access and download the entire database?

PS. I deploy the cluster using HELM in minikube.

---

Here are the YAML files:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-config
  labels:
    app: postgres
data:
  POSTGRES_DB: database-pg
  POSTGRES_USER: postgres
  POSTGRES_PASSWORD: postgres
  PGDATA: /data/pgdata
---
kind: PersistentVolume
apiVersion: v1
metadata:
  name: postgres-pv-volume
  labels:
    type: local
    app: postgres
spec:
  storageClassName: manual
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteMany
  hostPath:
    path: "/mnt/data"
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: postgres-pv-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
  labels:
    app: postgres
spec:
  ports:
    - name: postgres
      port: 5432
      nodePort: 30432
  type: NodePort
  selector:
    app: postgres
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: postgres-service
  selector:
    matchLabels:
      app: postgres
  replicas: 2
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:13.2
          volumeMounts:
            - name: postgres-disk
              mountPath: /data
          # Config from ConfigMap
          envFrom:
            - configMapRef:
                name: postgres-config
  volumeClaimTemplates:
    - metadata:
        name: postgres-disk
      spec:
        accessModes: ["ReadWriteOnce"]
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  labels:
    app: postgres
spec:
  selector:
    matchLabels:
      app: postgres
  replicas: 2
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:13.2
          imagePullPolicy: IfNotPresent
          envFrom:
            - configMapRef:
                name: postgres-config
          volumeMounts:
            - mountPath: /var/lib/postgresql/data
              name: postgredb
      volumes:
        - name: postgredb
          persistentVolumeClaim:
            claimName: postgres-pv-claim
---

Answer 1

I found a solution to my problem of downloading the volume directory, however when I run multiple replicasets of postgres, the tables of the DB are still scattered between the pods.

Here's what I did to download the postgres volume:

First of all, minikube supports some specific directories for volume appear:

minikube is configured to persist files stored under the following directories, which are made in the Minikube VM (or on your localhost if running on bare metal). You may lose data from other directories on reboots.

 /data /var/lib/minikube /var/lib/docker /tmp/hostpath_pv /tmp/hostpath-provisioner

So I've changed the mount path to be under the /data directory. This made the database volume visible.

After this I ssh'ed into minikube and copied the database volume to a new directory (I used /home/docker as the user of minikube is docker ).

sudo cp -R /data/pgdata /home/docker

The volume pgdata was still owned by root (access denied error) so I changed it to be owned by docker . For this I also set a new password which I knew:

sudo passwd docker # change password for docker user
sudo chown -R docker: /home/docker/pgdata # change owner from root to docker

Then you can exit and copy the directory into you local machine:

exit
scp -r $(minikube ssh-key) docker@$(minikube ip):/home/docker/pgdata [your_local_path].

NOTE

Mario's advice, which is to use pgdump is probably a better solution to copy a database. I still wanted to download the volume directory to see if it has the full database, when the pods have only a part of all the tables. In the end it turned out it doesn't.