python flask scheduling: all workers start scheduler task

Question

I use scheduled tasks inside a flask application:

    import time
    import atexit

    from apscheduler.schedulers.background import BackgroundScheduler


    def intervall():
        with app.app_context():
            call_function_to_do()

    scheduler = BackgroundScheduler()
    scheduler.add_job(func=intervall, trigger="interval", seconds=5)
    scheduler.start()

    # Shut down the scheduler when exiting the app
    atexit.register(lambda: scheduler.shutdown())

My problem is that all gunicorn workers are starting the scheduler task. In this case checking for a timeout situation and sending an email to a user.

Does anybody have an idea how to only have one worker excute call_function_to_do() ?

Answer 1

There are basically two options. One of them outlined in the FAQ is to separate scheduler into separate process and communicate with the process using some remote access protocol (eg using RPyC ). The other method is to start the scheduler in Flask app, but ensure that only one such instance is run. You can use some kind of locking mechanism, eg RedLock .