Mount volume with docker-compose for Jupyter and RStudio instance

Question

I am learning Docker and managed to create a RStudio instance and a Jupyter Notebook instance using docker-compose building an image. While I can log in and create scripts from the instances, I noticed these scripts are not saved "persistently", and I cannot find them in my main folders /home/rstudio_scripts and home/jupyter_scripts .

I have learned that in order for the scripts/data created in these instances to be "persistent" even after docker containers are down, you need to mount volumes.

So I have tried to mount volumes and create a specific folder for RStudio & Jupyter in the docker-compose.yml below:

But there is clearly something wrong as neither the folders nor the data appear.

My docker-compose.yml and the Dockerfile for RStudio are both in a folder called Docker , that is within this folder that I am building the image docker build -t general_docker.

I would also like that any data created either in the Rstudio or Jupyter instance can be read/write by both instances, but I don't know if I should use something like chmod 777... after, when the containers are running.

Any help much appreciated !

EDIT 1 : Let me attach the updated docker-compose.yml. The following allow me to have a new folder I created in the home directory called R_and_Jupyter_scripts containing all my scripts, to be accessible in my Rstudio instance as well as my Jupyter notebook. But I would like that any new script created from the Jupyter or RStudio instance does not disappear after doing 'docker-compose down'. What part of the docker-compose.yml should I change?

version: "3.5"
services:
  rstudio:
    environment:
      - USER=username
      - PASSWORD=password
    image: "rocker/tidyverse:latest"
    build:
     context: ./
     dockerfile: Dockerfile
    volumes:
      - $HOME/R_and_Jupyter_scripts:/home/rstudio/r_scripts
    container_name: rstudio
    ports:
     - 8787:8787

  jupyter:
    image: 'jupyter/datascience-notebook:latest'
    ports:
     - 8888:8888
    volumes:
      - $HOME/R_and_Jupyter_scripts:/home/jovyan/work
    container_name: jupyter

EDIT 2:

I have edited the code above to write in volumes only absolute paths (see below) but still does not do what I need.

I have created a text file output.txt from RStudio and when I do sudo find / -name "output.txt" I expect this file to be in /home/ec2-user/R_and_Jupyter_scripts but it ends up being in two different places that look like folders that are still in the container itself:

/var/lib/docker/overlay2/66513c53c04786298cac012ea032be58d434131ce04e73f75bf63ca1d0e358d6/diff/home/maxence/r_scripts/output.txt

and

/var/lib/docker/overlay2/66513c53c04786298cac012ea032be58d434131ce04e73f75bf63ca1d0e358d6/merged/home/maxence/r_scripts/output.txt

docker-compose.yml:

version: "3.5"
services:
  rstudio:
    environment:
      - USER=username
      - PASSWORD=password
    image: "rocker/tidyverse:latest"
    build:
     context: ./
     dockerfile: Dockerfile
    volumes:
      - /home/ec2-user/R_and_Jupyter_scripts:/home/rstudio/r_scripts
    container_name: rstudio
    ports:
     - 8787:8787

  jupyter:
    image: 'jupyter/datascience-notebook:latest'
    ports:
     - 8888:8888
    volumes:
      - /home/ec2-user/R_and_Jupyter_scripts:/home/jovyan/work
    container_name: jupyter

Answer 1

The volume definition creates folders inside the container. If you want to mount a folder on your Docker host, you must give the full path to that host folder; just the relative local folder name is not sufficient.

The config you posted is using a "named volume", instead of mounting a host volume. It acts close enough to a disk image; Docker will have a reference to a {PROJECT}_Docker volume when you run docker volume ls . This saves the contents of the folder elsewhere, not normally accessible except via mounting in other Docker containers.

If you want to host volume mount, remove the volumes: section at the end of the Compose file, and instead have in each service definition something like

volumes:
  - /home/ML_Enthousiast/my_project/Docker:/home/rstudio_scripts