I am looking to run a dockerized python script once every minute and I would like to understand how to make this script run every minute and would I do it via cron?
It will be collecting data via an api call and loading to a mysql db in another docker container I have set up, but for now as an example here is what I have running:
Dockerfile:
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD [ "python", "./loading_data.py" ]
loading_data.py:
print('i am loading data to mysql...')
I have my docker python image built:
REPOSITORY TAG IMAGE ID CREATED SIZE
docker_python latest f7222d16bbce 3 minutes ago 938MB
If I run the image it will execute the printing statement upon being built correct? So do I want to:
write into the python script an infinite loop and have my OS start the docker container and restart it if it fails? (feels wrong)
do I want to have a cron job start the container, destroy it and restart it every minute? (feels excessive)
Or is there a simpler more elegant and better way?
-------------------------------------------------- Attempt:
I've been trying to setup the python script as a cron job as suggested following https://github.com/cheyer/docker-cron example here is my build:
Dockerfile:
#python install----------
FROM python:3.6
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD [ "python", "./loading_data.py" ]
#cron install----------
FROM ubuntu:latest
# Install cron
RUN apt-get update
RUN apt-get install cron
# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
# Add shell script and grant execution rights
ADD script.sh /script.sh
RUN chmod +x /script.sh
# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
# Create the log file to be able to run tail
RUN touch /var/log/cron.log
# Run the command on container startup
CMD cron && tail -f /var/log/cron.log
script.sh:
python ./loading_data.py >> /var/log/cron.log 2>&1
crontab:
* * * * * root /script.sh
# An empty line is required at the end of this file for a valid cron file.
Built and ran the container and the cron job runs no problem every minute, but python can't be found within the container. How would I go about repairing this issue? I believe I am very close:
sudo docker exec -i -t 3c54ffaf2674db1ab0751abe93ce77956d8b5b594a7c48cc589c4841761d4e71 /bin/bash
root@3c54ffaf2674:/# cat /var/log/cron.log
/script.sh: 1: /script.sh: python: not found
/script.sh: 1: /script.sh: python: not found
/script.sh: 1: /script.sh: python: not found
As you already listed down the solutions and their pros and cons. 1 way in which I would be doing is, write a corn job which will run the script "python ./loading_data.py"
every minute and place it into the same container which means container is always running and it will run the cornjob as well inside.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.