How do I create a persistent volume in docker for my postgresql database?

Question

I am creating a docker image of a Django server that uses a postgresql database to keep track of login information, uploaded/downloaded files, etc. I want the data in my database to stick around even when I exit and relaunch the docker image.

I created a docker-compose file and entrypoint file that successfully launches the server docker image and creates a volume for the database. Every tutorial that I've read online has said that having persistent data is as simple as mapping your database volume from the location you want it stored on the host to the /var/lib/postgresql/data location in the docker image.

docker-compose.yml:

version: '3.7'

services:
  web:
    build: ./
    command: python manage.py runserver 0.0.0.0:8000
    ports:
      - 8000:8000
    environment:
      - DEBUG=1
      - SECRET_KEY=foo
      - SQL_ENGINE=django.db.backends.postgresql
      - SQL_HOST=db
      - SQL_PORT=5432
      - DATABASE=postgres
    depends_on:
      - db
  db:
    image: postgres
    volumes:
      - ./postgres_data:/var/lib/postgresql/data/
    environment:
      ...
    ports:
      - 5432:5432

volumes:
  postgres_data:

docker-entrypoint.sh:

#!/bin/sh

if [ "$DATABASE" = "postgres" ]
then
    echo "Waiting for postgres..."

    while ! nc -z $SQL_HOST $SQL_PORT; do
      sleep 0.1
    done

    echo "PostgreSQL started"
fi

python manage.py flush --no-input
python manage.py migrate

exec $@

Dockerfile:

FROM python:3.7.4-alpine

ARG PROJECT=MyProject
ARG PROJECT_DIR=/var/www/${PROJECT}
RUN mkdir -p $PROJECT_DIR
WORKDIR $PROJECT_DIR

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

RUN apk update \
    && apk add --virtual build-deps gcc python3-dev musl-dev \
    && apk add postgresql-dev \
    && pip install psycopg2 \
    && pip install django \
    && pip install djangorestframework \
    && pip install django-sslserver \
    && pip install djangosecure \
    && pip install psycopg2 \
    && pip install django-environ \
    && apk del build-deps

COPY . $PROJECT_DIR/

ENTRYPOINT ["/var/www/EchoNine/docker-entrypoint.sh"]

settings.py

import os
import sys
from django.conf import settings
import environ

env = environ.Env(
    # set casting, default value
    DEBUG=(bool, False)
)

BASE_DIR = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

MEDIA_URL = 'inspections/'

TEST_ROOT = os.path.join(BASE_DIR, 'test')

if ('test' in sys.argv):
    MEDIA_ROOT = os.path.join(TEST_ROOT, 'inspections')
else:
    MEDIA_ROOT = os.path.join(BASE_DIR, 'inspections')

root = environ.Path(__file__) - 3 # three folder back (/a/b/c/ - 3 = /)
environ.Env.read_env(env.str('ENV_PATH', '.env.production')) # reading .env fil

SECRET_KEY = env('SECRET_KEY')

DEBUG = env('DEBUG') # This is set to False

ALLOWED_HOSTS = []

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'djangosecure',
    'sslserver',

    # Third-Party Apps
    'rest_framework',
    'rest_framework.authtoken',

    # Local Apps (Your project's apps)
    'MyProject.MyProject',
]

REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'rest_framework.authentication.TokenAuthentication',
    ],
}

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

MIDDLEWARE_CLASSES = (
    'djangosecure.middleware.SecurityMiddleware',
)

ROOT_URLCONF = 'MyProject.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'MyProject.wsgi.application'

DATABASES = {
    'default': {
        'ENGINE': os.environ.get('SQL_ENGINE', 'django.db.backends.sqlite3'),
        'NAME': os.environ.get('SQL_DATABASE', os.path.join(BASE_DIR, 'db.sqlite3')),
        'USER': os.environ.get('SQL_USER', 'user'),
        'PASSWORD': os.environ.get('SQL_PASSWORD', 'password'),
        'HOST': os.environ.get('SQL_HOST', 'localhost'),
        'PORT': os.environ.get('SQL_PORT', '5432'),
    }
}

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True

SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True

STATIC_URL = '/static/'

When I launch the docker image, create a new super user inside the docker image (which stores data in the database), quit the docker image, and then relaunch the image, all database tables are empty again. I would expect the database to retain the database entries from the last run.

Both./postgres_data on the host and /var/lib/postgresql/data inside the docker image seem to be initialized with files and a folder structure in them.

Can anyone tell me what I'm doing wrong? Thanks.

Answer 1

you delete everything by your Entrypoints since the entrypoint will start with every container operation (start/restart)

PS : manage.py flush will delete all your Data