How to pass arguments to scoring file when deploying a Model in AzureML

Question

I am deploying a trained model to an ACI endpoint on Azure Machine Learning, using the Python SDK. I have created my score.py file, but I would like that file to be called with an argument being passed (just like with a training file) that I can interpret using argparse . However, I don't seem to find how I can pass arguments This is the code I have to create the InferenceConfig environment and which obviously does not work. Should I fall back on using the extra Docker file steps or so?

from azureml.core.conda_dependencies import CondaDependencies
from azureml.core.environment import Environment
from azureml.core.model import InferenceConfig

env = Environment('my_hosted_environment')
env.python.conda_dependencies = CondaDependencies.create(
    conda_packages=['scikit-learn'],
    pip_packages=['azureml-defaults'])
scoring_script = 'score.py --model_name ' + model_name
inference_config = InferenceConfig(entry_script=scoring_script, environment=env)

Adding the score.py for reference on how I'd love to use the arguments in that script:

#removed imports
import argparse

def init():
    global model

    parser = argparse.ArgumentParser(description="Load sklearn model")
    parser.add_argument('--model_name', dest="model_name", required=True)
    args, _ = parser.parse_known_args()

    model_path = Model.get_model_path(model_name=args.model_name)
    model = joblib.load(model_path)

def run(raw_data):
    try:
        data = json.loads(raw_data)['data']
        data = np.array(data)
        result = model.predict(data)
        return result.tolist()

    except Exception as e:
        result = str(e)
        return result

Interested to hear your thoughts

Answer 1

This question is a year old. Providing a solution to help those who may still be looking for an answer. My answer to a similar question is here . You may pass native python datatype variables into the inference config and access them as environment variables within the scoring script.

Answer 2

I tackled this problem differently. I could not find a (proper and easy to follow) way to pass arguments for score.py , when it is consumed by InferenceConfig . Instead, what I did was following 4 steps:

1. Created score_template.py and define variables which should be assigned
2. Read content of score_template.py and modify it by replacing variables with desired values
3. Write modified contents into score.py
4. Finally pass score.py to InferenceConfig

STEP 1 in score_template.py:

import json
from azureml.core.model import Model
import os
import joblib
import pandas as pd
import numpy as np
def init():
    global model
    #model = joblib.load('recommender.pkl')
    model_name="#MODEL_NAME#"
    model_saved_file='#MODEL_SAVED_FILE#'
    try: 
        model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), model_saved_file)
        model = joblib.load(model_path)
    except:
        model_path = Model.get_model_path(model_name)
        model = joblib.load(model_path)

def run(raw_data):
    try:
        #data=pd.json_normalize(data) 
        #data=np.array(data['data'])
        data = json.loads(raw_data)["data"]
        data = np.array(data)
        result = model.predict(data)
        # you can return any datatype as long as it is JSON-serializable
        return {"result": result.tolist()}
    except Exception as e:
        error = str(e)
        #error= data
        return error

STEP 2-4 in deploy_model.py:

#--Modify Entry Script/Pass Model Name--
entry_script="score.py"
entry_script_temp="score_template.py"

# Read in the entry script template
print("Prepare Entry Script")
with open(entry_script_temp, 'r') as file :
  entry_script_contents = file.read()

# Replace the target string
entry_script_contents = entry_script_contents.replace('#MODEL_NAME#', model_name)
entry_script_contents = entry_script_contents.replace('#MODEL_SAVED_FILE#', model_file_name)
# Write the file to entry script
with open(entry_script, 'w') as file:
  file.write(entry_script_contents)

#--Define configs for the deployment---
print("Get Environtment")
env = Environment.get(workspace=ws, name=env_name)
env.inferencing_stack_version = "latest"
print("Inference Configuration")
inference_config = InferenceConfig(entry_script=entry_script, environment=env, source_directory=base_path)
aci_config = AciWebservice.deploy_configuration(cpu_cores = int(cpu_cores), memory_gb = int(memory_gb),location=location)

#--Deloy the service---
print("Deploy Model")
print("model version:", model_artifact.version)
service = Model.deploy( workspace=ws,
                        name=service_name,
                        models=[model_artifact],
                        inference_config=inference_config,
                        deployment_config=aci_config,
                        overwrite=True )

service.wait_for_deployment(show_output=True) 

Answer 3

How to deploy using environments can be found here model-register-and-deploy.ipynb . InferenceConfig class accepts source_directory and entry_script parameters , where source_directory is a path to the folder that contains all files(score.py and any other additional files) to create the image.

This multi-model-register-and-deploy.ipynb has code snippets on how to create InferenceConfig with source_directory and entry_script.

from azureml.core.webservice import Webservice
from azureml.core.model import InferenceConfig
from azureml.core.environment import Environment

myenv = Environment.from_conda_specification(name="myenv", file_path="myenv.yml")
inference_config = InferenceConfig(entry_script="score.py", environment=myenv)

service = Model.deploy(workspace=ws,
                       name='sklearn-mnist-svc',
                       models=[model], 
                       inference_config=inference_config,
                       deployment_config=aciconfig)

service.wait_for_deployment(show_output=True)

print(service.scoring_uri)