Restarting The Jobs In ApScheduler Python When the Wsgi Server is Restarted

Question

I'm using python Apscheduler to schedule my jobs. All my jobs are stored as a cron job and use the BackgroundScheduler. I've the following codes:

def letschedule():
    jobstores = {
        'default': SQLAlchemyJobStore(url=app_jobs_store)
    }
    executors = {
        'default': ThreadPoolExecutor(20),
        'processpool': ProcessPoolExecutor(5)
    }
    job_defaults = {
        'coalesce': False,
        'max_instances': 1,
        'misfire_grace_time':1200
    }
    scheduler = BackgroundScheduler(jobstores=jobstores, executors=executors, job_defaults=job_defaults, timezone=utc)
    #jobstores=jobstores, executors=executors, job_defaults=job_defaults, timezone=utc
    return scheduler

And I start the job scheduler as follow in the app:

sch = letschedule()
sch.start()
log.info('the scheduler started')

And I've the following add job function.

def addjobs():


    jobs = []
    try:
        sch.add_job(forecast_jobs, 'cron', day_of_week=os.environ.get("FORECAST_WEEKOFDAY"),  
            id="forecast",
             replace_existing=False,week='1-53',hour=os.environ.get("FORECAST_HOUR"), 
             minute=os.environ.get("FORECAST_MINUTE"), timezone='UTC')
        jobs.append({'job_id':'forecast', 'type':'weekly'})

        log.info('the forecast added to the scheduler')
    except BaseException as e:
        log.info(e)
        pass
    try:
        sch.add_job(insertcwhstock, 'cron', 
            id="cwhstock_data", day_of_week='0-6', replace_existing=False,hour=os.environ.get("CWHSTOCK_HOUR"),
            minute=os.environ.get("CWHSTOCK_MINUTE"),
            week='1-53',timezone='UTC')
        jobs.append({'job_id':'cwhstock_data', 'type':'daily'})
        log.info('the cwhstock job added to the scheduler')
    except BaseException as e:
        log.info(e)
        pass
    return json.dumps({'data':jobs})

I use this in the flask application, I call the /activatejobs and the jobs are added to the scheduler and it works fine. However when I restart the wsgi server, the jobs aren't started again, I've to remove the .sqlite file and add the jobs again. What I want is the jobs are supposed to be restarted automatically once the scheduler is started (if there are already jobs in the database.) I tried to get such result trying some ways, but couldn't. Any help would be greatly appreciated. Thanks in Advance.

Answer 1

I also had the same problem using FastApi framework. I could solve the problem after add this code to my app.py:

scheduler = BackgroundScheduler()
pg_job_store = SQLAlchemyJobStore(engine=my_engine)
scheduler.add_jobstore(jobstore=pg_job_store, alias='sqlalchemy')
scheduler.start()

Adding this code, after I restart the application server I could see apscheduler logs searching for jobs:

2021-10-20 14:37:53,433 - apscheduler.scheduler - INFO     => Scheduler started
2021-10-20 14:37:53,433 - apscheduler.scheduler - DEBUG    => Looking for jobs to run
Jobstore default:
    No scheduled jobs
Jobstore sqlalchemy:
    remove_from_db_job (trigger: date[2021-10-20 14:38:00 -03], next run at: 2021-10-20 14:38:00 -03)
2021-10-20 14:37:53,443 - apscheduler.scheduler - DEBUG    => Next wakeup is due at 2021-10-20 14:38:00-03:00 (in 6.565892 seconds)

It works for me.