Python-Flask setting up a que if a process is already running

Question

I have a python flask application where a person is gonna request data by sending a query from A form. This data goes through a certain python script that does some api-requests and covert the data into a geographic standard.

The thing is because this data can take some time based on how many datapoints there are, this will happen in the background (we are researching this for Azure). Also there is another problem, and that is cueing up. Because if one request is running, another one cannot be started up. And the last command cannot be saved:

@app.route('/handle_data', methods=['POST'])
def handle_data():
    sper_year = int(request.form["Speryear"])
    email = request.form["inputEmail"]
    url = request.form["api-url-input"]
    random_string=get_random_string(5)
    # app.route('/request-completed')
    Requested_data = Program_Converter.main(url, sper_year,random_string)

Requested_data = Program_Converter.main(url, sper_year,random_string) is the function that needs to be qued.

How do I do this?

Answer 1

I believe that the most recommended way is to run this task asynchronously. Take a look at Celery and pick up a backend (I recommend Redis), with this setup you can provide Celery a task that will run your GIPOD_Converter process in the background and store the result somewhere you chose, then it can be sent back to the user.

Note that Celery will provide you a task id and is up to your client (web interface or mobile app, I'm not sure what you're working with) to poll and endpoint and wait for the celery task to come to an end.

There are a couple of examples on who to implement this all over the web, but take a look at the Flask official documentation and check out this article from the mighty Miguel Grinberg , I believe those are the best starting points for you.