Repack inference model with requirements.txt inside source_dir without installing them during the process in SageMaker

Question

After training a custom model, i need to create an inference model and then deploy the relevant endpoint.

When, in the execution of the pipeline, i have to inject a custom inference script, a model repacking process is triggered. The inference model needs to have the requirements.txt file (the same as the trained model).

When the repacking process is started, a default machine ml.m5.large with the training image sagemaker-scikit-learn:0.23-1-cpu-py3 is instantiated. If the requirements.txt file is present in the inference code folder, this process will try to install the packages (although it is not necessary, should be a simple repacking of a tar.gz.).

Unfortunately, having specified particular library versions, it will fail.

For example:

ERROR: Ignored the following versions that require a different python version: 1.22.0 Requires-Python >=3.8; 1.22.0rc1 Requires-Python >=3.8; 1.22. 0rc2 Requires-Python >=3.8; 1.22.0rc3 Requires-Python >=3.8; 1.22.1 Requires-Python >=3.8; 1.22.2 Requires-Python >=3.8; 1.22.3 Requires-Python >=3. 8; 1.22.4 Requires-Python >=3.8; 1.23.0 Requires-Python >=3.8; 1.23.0rc1 Requires-Python >=3.8; 1.23.0rc2 Requires-Python >=3.8; 1.23. 0rc3 Requires-Python >=3.8; 1.23.1 Requires-Python >=3.8; 1.23.2 Requires-Python >=3.8; 1.23.3 Requires-Python >=3.8; 1.23.4 Requires-Python >=3.8
ERROR: Could not find a version that satisfies the requirement numpy==1.23.0

This is the code I'm running:

inf_img_uri = sagemaker.image_uris.retrieve(
    framework='pytorch',
    region=region,
    image_scope='inference',
    version="1.12.0",
    instance_type='ml.m5.xlarge',
    py_version='py38'
)

pytorch_model = Model(
    image_uri=inf_img_uri,
    model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts,
    role=role,
    entry_point='inference.py',
    sagemaker_session=PipelineSession(),
    source_dir=os.path.join(LIB_DIR, "test"),  # here is inference.py and requirements.txt
    name=model_name,
)

step_create_model = ModelStep(
    name="infTest",
    step_args=pytorch_model.create(instance_type="ml.m5.xlarge"),
    description = 'Create model for inference'
)

Is there any way to prevent the template repack from trying to install packages from the requirements.txt ?

My current solution : I have omitted the file in the directory and manually install the packages with subprocess.check_call([sys.executable, "-m", "pip", "install", package]) in the inference code. But I find this approach wrong for Batch Inference processes (since it would be executed every time) and also inconsistent.

Answer 1

The SageMaker SDK will always repackage the tar ball to include the inference.py script and then re-upload the tar ball to S3.

In general, SageMaker Framework containers will install the packages specified in the requirements.txt file.

If you do not want this to occur you can leave out the requirements.txt file and extend the sagemaker-scikit-learn:0.23-1-cpu-py container to include all the necessary dependencies. That way the packages will be baked into the image and every time you kick off a Batch Transform Job the packages will not be installed again.

https://docs.aws.amazon.com/sagemaker/latest/dg/prebuilt-containers-extend.html