How to store a .tar.gz formatted model to AWS SageMaker and use it as a deployed model?

Question

I have a pre-trained BERT model which was trained on Google Cloud Platform, and the model is stored in a .tar.gz formatted file, I wanted to deploy this model to SageMaker and also be able to trigger the model via API, how can I achieve this?

I found this question is a little bit related to what I'm asking here, but it's for a scikit-learn model, I'm new to this area, can someone give me some guidance regarding this? Many thanks.

Answer 1

To use an existing model object this just needs to be packaged in a .tar.gz and stored in S3.

Once that is packaged then you can deploy it using the sage maker API through sage maker notebooks \\studio or terraform.

Packaging the code

File Structure

Here is the structure of what the .tar.gz should follow.

|----- preprocessing.joblib # multiple model objects included here
|----- model.joblib
|----- code
    |----- inference.py # used for custom transformation \ loading model etc. 
    |----- requirements.txt # can be ran to include dependencies not in the docker image

Zipping your files

ex.

tar czvf model.tar.gz. model_xgb.joblib preprocessor.joblib code

Deploying Endpoint

Once you have the model packaged then you can create an endpoint by loading the model into one of the model classes and then deploying it.

from sagemaker.amazon.amazon_estimator import get_image_uri
from sagemaker.model import Model
from sagemaker.pipeline import PipelineModel

container=sagemaker.image_uris.retrieve("xgboost", boto3.Session().region_name, "0.90-1")
print(container)

model_name = 'some_model_123'
model_location= "s3://somebucket/somemodelpath/someversion/model.tar.gz"

env = {
    'SAGEMAKER_REQUIREMENTS': 'requirements.txt',
    'SAGEMAKER_PROGRAM': 'inference.py',
    'SAGEMAKER_SUBMIT_DIRECTORY' : model_location
    }

model= Model(
    model_data=model_location, 
    image_uri =container,
    env = env
)

endpoint_name = model_name

pipeline_model = PipelineModel(name=model_name,
                               role=role,
                               models=[
                                    model
                               ])

pm = pipeline_model.deploy(initial_instance_count=1, instance_type="ml.c4.xlarge", endpoint_name=endpoint_name)

References

Here is a reference of a bring your own model which helped me a bunch. https://medium.com/geekculture/84af8989d065

Here is a reference to a similar question that I posted while struggling with as well. How to handle custom transformation/ inference and requirements in sagemaker endpoints