Crontab for Python script every 5 minutes

Question

I have a Python script which I want to launch every 5 minutes despite the user who launchs it. My idea is to generate a code which can be downloaded from a repository and used by anyone, so I don't want to specify any user in the crontab as I don't know what will be its name.

I have a Luigi pipeline which makes a comprobation; if all the requirements are met it performs its function, and if they aren't it does nothing. I want to run the pipeline each 5 minutes to check if the conditions have changed.

I have written this script named my_cron.py

from crontab import CronTab

class CronManager:
    def __init__(self):
        self.cron = CronTab()

    def add_minutely(self, command, interval, environment=None):
        cron_job = self.cron.new(command=command)
        cron_job.minute.every(interval)
        cron_job.enable()
        self.cron.write('my_cron.tab')

if __name__ == '__main__':
    my_cron = CronManager()
    command = "python -m luigi --local-scheduler --module tasks Main"
    my_cron.add_minutely(command, 5)

The script I want to run each 5 minutes is python -m luigi --local-scheduler --module tasks Main which is on the same folder as my_cron.py and I would like that when I run python my_cron.py it displays on the same console I launched it the results of every 5 minute task.

However when I launch my_cron.py it just finish the process and I get this on my_cron.tab

*/5 * * * * python -m luigi --local-scheduler --module tasks Main

Answer 1

This looks like expected behaviour, are you expecting the crontab to be picked up and used by the system's cron process? If so your new crontab file needs to be saved somewhere where it will be processed by the system. Of course if there is no active cron process on the system, nothing will happen.

On a Unix system, you can use self.cron.write_to_user(user=True) . Full manual for python-crontab with all the information about how the various write functions is available here: https://pypi.org/project/python-crontab/