Sagemaker：指定自定义入口点会导致未找到错误

Question

I am trying to deploy my object detection model that was trained using tensorflow to sagemaker. I was able to deploy it without specifying any entry points during model creation but it turns out doing that will only work for small sizes images (Sagemaker has limit of 5MB). The code I used for this is as:

from sagemaker.tensorflow.serving import Model

# Initialize model ...
model = Model(
    model_data= s3_path_for_model,
    role=sagemaker_role,
    framework_version="1.14",
    env=env)

# Deploy model ...
predictor = model.deploy(initial_instance_count=1,
                         instance_type='ml.t2.medium')


# Test using an image ...
import cv2
import numpy as np

image_content = cv2.imread("PATH_TO_IMAGE",
                           1).astype('uint8').tolist()
body = {"instances": [{"inputs": image_content}]}

# Works fine for small images ...
# I get predictions perfectly with this ...
results = predictor.predict(body)

So, I googled around and found that I need to pass an entry_point for Model() in order to predict for larger images. Something like:

model = Model(
        entry_point="inference.py",
        dependencies=["requirements.txt"],
        model_data=  s3_path_for_model,
        role=sagemaker_role,
        framework_version="1.14",
        env=env
)

But doing this gives FileNotFoundError: [Errno 2] No such file or directory: 'inference.py' . A little help here please. I am using sagemaker-python-sdk . My folder structure is as:

model
    |__ 001
          |__saved_model.pb
          |__variables
                |__<contents here>

    |__ code
          |__inference.py
          |__requirements.txt

Note: I have also tried, ./code/inference.py and /code/inference.py.

Answer 1

5MB is a hard limit for real-time endpoints.

Are you sure you need to pass such large images for prediction? Most use cases work fine with smaller, lower resolution images.

If you need real-time prediction, one workaround would be to pass the image S3 URI in the prediction request (instead of the image itself), and load the image from S3.

If you don't need real-time prediction, you should look at batch transform, which doesn't enforce that size restriction: https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html