cannot load locally pretrained model that I deployed via sagemaker notebook

Question

Trying to deploy locally pretrained model via sagemaker to make a endpoint and use it I have been learning AI/ML field lately, and I am such a noob. Need a help.

I deployed a model

from sagemaker.pytorch import PyTorchModel

pytorch_model = PyTorchModel(model_data='model.tar.gz',
                             role=role,
                             entry_point='inference.py',
                             framework_version="1.11.0",
                             py_version="py38")

predictor = 
pytorch_model.deploy(instance_type='ml.g4dn.xlarge', 
initial_instance_count=1)

and predict data

from PIL import Image 
data = Image.open('./samples/inputs/1.jpg')
result = predictor.predict(data)
img = Image.open(result)
img.show()

as a result I got an error that I cannot load my model

you can see the error log as well in here

Answer 1

I didn't make it fully, but loading model is working well now. It was due to structure of model.tar.gz and inference.py.

my model.tar.gz is structured like below

model.tar.gz
|
|- model.pt
|- code/
| | | - code/inference.py
| | | - code/requirements.txt

This structure is required.

Sometimes, model.pt goes empty file somehow when you make a model.tar.gz, so it is better to check model.pt size is not "0" b4 make tar.gz file.

In addition, if you get wrong with inference.py, such as input_fn, predict_fn, and output_fn, it may occur Backend worker process died even if model_fn works properly in my experience.

I am struggling with request_body now. I keep get request_body as bytearray even if I input JSON file. my expectation was request_body had to be JSON, but its not.

I will update if I solve this problem.