How to start Docker containers from shell script based on arguments

Question

I have a python app that has the next structure:

./DB  
./lib  
./docker-compose.yml  
./Dockerfile  
./module1.py  
./module2.py  
./module3.py  
./app_run.sh  
./requirements.txt  

What I want to get

• Mysql server runs on a separate container.
• The app docker image.
• Modules' scripts are running separately as the containers from the app_run.sh.

My app image is about 1.5G, so what I want to achieve is have one app image with the ability to run multiple containers with the only certain module. So that I can run the container with module1.py script (or any other) or many of such scripts.

In such a way I hope to speed up the process as there will be no need to build an app image every time, only to connect to it.

Why do I need to run such containers from app shell script.
The shell script gets the needed parameters like URL and token from the external source that then are used in modules.

The shell script has the following structure:

#!/bin/sh

ip=$1
url=$2
token=$3

folder="./temp/"
folder_oss='./temp/DB/TEST/'
db_folder="./temp/DB"
app="./"

name=$(date '+%Y%m%d%H%M%S')

cd $folder && mkdir -p "$folder$name"
folder="$folder$name"
cd $app

if [ $ip = 1 ]; then
python3.6 ./module1.py -u $url -t $token -f $folder -d $db_folder 
 status=${PIPESTATUS};
elif [ $ip = 3 ]; then
python3.6 ./module2.py -u $url -t $token -f $folder |& tee "$folder/logtemp.txt"
status=${PIPESTATUS};
elif [ $ip = 7 ]; then
python3.6 ./module.py -u $url -t $token -f $folder -d $db_folder |& tee "$folder/load_logtemp.txt"
status=${PIPESTATUS};
else
   echo "Error while predicting"
   status=1;
fi  

I suppose that instead of python3.6./module1.py I should write something like docker container run. But how should I specify those modules as separate containers?

Currently, my Dockerfile looks like this:

FROM python:3.6
COPY . /app
WORKDIR /app
RUN pip freeze > requirements.txt
EXPOSE 7524
CMD ["./app_run.sh", "1","link/to/settings.json, "token"]  

And the docker_compose.yml:

version: "2"
services:
   app:
     build: .
     environment:
        - DB_HOST=mysql
        - DB_PORT=7524
     ports:
        - 8080:8080
     depends_on:
        - mysql

   mysql:
     image: mysql:latest
     restart: always
     environment:
        - MYSQL_USER=user
        - MYSQL_ROOT_PASSWORD=54321
     ports:
        - 7524:3303
     volumes:
       - my-db:/var/lib/mysql

 volumes:
    my-db:

Answer 1

Move app_run.sh to entrypoint and pass module name and IP, token etc to CMD . Your Dockerfile will look like

FROM python:3.6
COPY . /app
WORKDIR /app
EXPOSE 7524
RUN chmod +x app_run.sh
ENTRYPOINT ["./app_run.sh"]
CMD ["1","link/to/settings.json","token"] 

and then process these arguments in entrypoint in your case it's app_run.sh`

#!/bin/sh
ip=$1
url=$2
token=$3
echo "IP is $ip , URL is $url , token is $token"
mkdir temp
folder="./temp/"
folder_oss='./temp/DB/TEST/'
db_folder="./temp/DB"
app="./"

name=$(date '+%Y%m%d%H%M%S')

cd $folder
mkdir -p "$folder$name"
folder="$folder$name"
cd $app

if [ $ip = 1 ]; then
python3.6 ./module1.py -u $url -t $token -f $folder -d $db_folder 
 status=${PIPESTATUS};
elif [ $ip = 3 ]; then
python3.6 ./module2.py -u $url -t $token -f $folder  
status=${PIPESTATUS};
elif [ $ip = 7 ]; then
python3.6 ./module.py -u $url -t $token -f $folder -d $db_folder 
status=${PIPESTATUS};
else
   echo "Error while predicting"
   status=1;
fi 

remember one thing, if python running in the foreground and its long-running process then it status will not reachable status=${PIPESTATUS};

So if you want to run container using IP 3 , all you need

docker run -it --rm my_image "3" "link/to/settings.json" "token

by default, it will run with IP 1.

Answer 2

One option would be to override the command in docker-compose.yml (see https://docs.docker.com/compose/compose-file/compose-file-v2/#command )

You could even make it take and environment variable.

Something like:

services:
  app:
    ...
    command:
      - "./app_run.sh"
      - ${IP}
      - ${URL}
      - ${TOKEN}

You could then start a run with:

$ IP=1 URL="link/to/settings.json" TOKEN=token docker-compose up

It could also be beneficial to mount your moduleX.py files into the container, so you won't have to rebuild it if adding more modules. This would probably be easiest if those module files were in their own folder. Then you could use something like (in docker-compose.yml):

services:
  app:
    ...
    volumes: ./modules:/app/modules

See also docs on volumes: https://docs.docker.com/compose/compose-file/compose-file-v2/#volumes