Parallel computation with joblib in Flask

Question

I have a python function that I need to call repeatedly with different argument values. I would like to execute this in parallel across multiple CPUs. I have done this successfully using the joblib module. I would now like to make my code available as a web app using flask running on an AWS EC2 instance with multiple CPUs. Here is a toy example of what I have tried:

from flask import Flask
from joblib import Parallel, delayed
from time import sleep

def myfunc(x):
    sleep(5)
    return x

application = Flask(__name__)

@application.route('/', methods = ['GET'])
def getresult():
    out = Parallel(n_jobs=-1, verbose=10)(delayed(myfunc)(i) for i in range(5))
    return str(sum(out))

if __name__ == "__main__":
    application.debug = True
    application.run()

The problem is that this code does not run in parallel across multiple CPUs. I get the following warning and output (the elapsed times confirm that it is not running in parallel):

    /Library/anaconda/lib/python3.6/site-packages/joblib/parallel.py:547:
    UserWarning: Multiprocessing-backed parallel loops cannot be nested below 
    threads, setting n_jobs=1
      **self._backend_args)
    [Parallel(n_jobs=-1)]: Done   1 out of   1 | elapsed:    5.0s remaining:    0.0s
    [Parallel(n_jobs=-1)]: Done   2 out of   2 | elapsed:   10.0s remaining:    0.0s
    [Parallel(n_jobs=-1)]: Done   3 out of   3 | elapsed:   15.0s remaining:    0.0s
    [Parallel(n_jobs=-1)]: Done   4 out of   4 | elapsed:   20.0s remaining:    0.0s
    [Parallel(n_jobs=-1)]: Done   5 out of   5 | elapsed:   25.0s remaining:    0.0s
    [Parallel(n_jobs=-1)]: Done   5 out of   5 | elapsed:   25.0s finished

Any suggestions?

Answer 1

Look at the UserWarning you get there:

UserWarning: Multiprocessing-backed parallel loops cannot be nested below 
threads, setting n_jobs=1

Maybe this helps:

Multiprocessing backed parallel loops cannot be nested below threads, setting n_jobs=1

Flask is probably spinning up it's own threads under the hood, so your getresult() might not run in MainThread.