Deploying python machine learning model using Azure Functions

Question

I developed a machine learning model locally and wanted to deploy it as web service using Azure Functions.

At first model was save to binary using pickle module and then uploaded into blob storage associated with Azure Function (python language, http-trigger service) instance.

Then, after installing all required modules, following code was used to make a predicition on sample data:

import json
import pickle
import time

postreqdata = json.loads(open(os.environ['req']).read())

with open('D:/home/site/wwwroot/functionName/model_test.pkl', 'rb') as txt:
    mod  = txt.read()
txt.close()

model = pickle.loads(mod)
scs= [[postreqdata['var_1'],postreqdata['var_2']]]

prediciton = model.predict_proba(scs)[0][1]

response = open(os.environ['res'], 'w')
response.write(str(prediciton))
response.close()

where: predict_proba is a method of a trained model used for training and scs variable is definied for extracting particular values (values of variables for the model) from POST request.

Whole code works fine, predictions are send as a response, values are correct but the execution after sending a request lasts for 150 seconds! (locally it's less than 1s). Moreover, after trying to measure which part of a code takes so long: it's 10th line (pickle.loads(mod)).

Do you have any ideas why it takes such a huge amount of time? Model size is very small (few kB).

Thanks

Answer 1

When a call is made to the AML the first call must warm up the container. By default a web service has 20 containers. Each container is cold, and a cold container can cause a large(30 sec) delay.

I suggest you referring to this thread Azure Machine Learning Request Response latency and this article about Azure ML performance.

Hope it helps you.