Background worker process crashing

Question

I'm creating a web app that takes username as input and perform some tasks with it. I'm using flask for the web and heroku to deploy it.I've two main python scripts these run the app.app.py and task.py . Everything is okey into my app.py file, where i used flask code for making the app but into my task.py file I've a

Initialisation step

from instabot import Bot
bot = Bot()
bot.login(username="myusername", password="mypassword")

now I deployed these script into the Procfile like

worker: python task.py
web: gunicorn app:app

but after deploying when i check it's log I'm getting

logs

2022-01-28T18:30:06.376795+00:00 
heroku[worker.1]: Starting process 
with command `python task.py`
        2022-01- 
28T18:30:07.081356+00:00 
heroku[worker.1]: State changed 
from starting to up
        2022-01- 
28T18:30:08.213687+00:00 
heroku[worker.1]: Process exited 
with status 0
        2022-01- 
28T18:30:08.274498+00:00 
heroku[worker.1]: State changed 
from up to crashed

as you can see it's getting crashed !! i don't know what's my mistake is can you please help me to figure out : (

Answer 1

The code in my worker is almost stolen from the redis-rq documentation, with some changes to make it work also in our local development enviroment:

import os
from rq import Queue, Connection
from tools import get_redis_store

# we need to use a different worker when we are in Heroku
if 'DYNO' in os.environ:
    from rq.worker import HerokuWorker as Worker
else:
    from rq import Worker

redis_url = os.getenv('REDIS_URL')
if not redis_url:
    raise RuntimeError('Set up Redis first.')

listen = ['default']

conn = get_redis_store()

if __name__ == '__main__':
    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

On tools.py

_redis_store = None


def get_redis_store():
    '''
    Get a connection pool to redis based on the url configured
    on env variable REDIS_URL

    Returns
    -------
    redis.ConnectionPool
    '''
    global _redis_store
    if not _redis_store:
        redis_url = os.getenv('REDIS_URL')
        if redis_url:
            logger.debug('starting redis: %s ' % redis_url)
            if redis_url.startswith('rediss://'):
                # redis 6 encripted connection
                # heroku needs disable ssl verification
                _redis_store = redis.from_url(
                    redis_url, ssl_cert_reqs=None)
            else:
                _redis_store = redis.from_url(redis_url)
        else:
            logger.debug('redis not configured')

    return _redis_store

On the main app, to queue a task:

from rq import Queue
from tools import get_redis_store

redis_store = get_redis_store()

queue = Queue(connection=redis_store)

def _slow_task(param1, param2, paramn):
    # here I execute the slow task
    my_slow_code(param1, param2)

# when I need to execute the slow task
queue.enqueue(_slow_task, param1, param2)

There are multiple options to use redis on Heroku, depending on what you use, you could need to change the env variable REDIS_URL. The environment variable DYNO is checked to know if the code is running on Heroku or in a developer machine.