How to run a background task in Heroku?

Question

I have already build aa python app and deployed it using Flask and Python . Here is my Skelton of my code.

#app.py
@app.route('/', methods=['GET'])
def login():
    '''login process'''

@app.route('/reset-password', methods=['GET'])
def reset_password():
    '''reset password process'''

@app.route('/add-psa', methods=['GET'])
def add_user():
    '''add user process'''

if __name__ == '__main__':
    app.debug = True
    app.run(use_reloader=False, threaded=True)

Deployed app work fine in the Heroku. But sometime it takes more than 30 seconds to response which means H12 error according to Heroku doc . So I followed this tutorial to run background job. Here is what I've done so far

#worker.py
import os

import redis
from rq import Worker, Queue, Connection

listen = ['high', 'default', 'low']

redis_url = os.getenv('REDISTOGO_URL', 'redis://localhost:6379')

conn = redis.from_url(redis_url)

if __name__ == '__main__':
    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

Next...

#utils.py
import requests

def count_words_at_url(url):
    resp = requests.get(url)
    return len(resp.text.split())

I also made change in the Procfile and requirements.txt

I want to run my reset_password() in background since its taking more than 30seconds . Any one who can help me on this?

Answer 1

A solution is to spawn a thread from the web request: in this case the response can be returned (almost) immediately while a background thread starts and performs the necessary task (without the constraint of the HTTP timeout)

# my background thread
class MyWorker():

  def __init__(self, message):
    self.message = message

    thread = threading.Thread(target=self.run, args=())
    thread.daemon = True
    thread.start()

  def run(self):
    logging.info(f'run MyWorker with parameter {self.message}')

    # do something

The web request creates an instance of the thread and could inform the caller something is in progress (or apply a different workflow/message)

#app.py
@app.route('/reset-password', methods=['GET'])
def reset_password():
'''reset password process'''
   MyWorker('param_value')
   return "In progress... You will receive an email"