[英]Sagemaker timing out for flask model deployment
Below is the predict.py in the ECR Container .下面是 ECR Container 中的 predict.py。 Sagemaker endpoint gives "Status:Failed" output after retrying for 10-12 minutes.
Sagemaker 端点在重试 10-12 分钟后给出“状态:失败”输出。 Both /ping and /invocations methods are available
/ping 和 /invocations 方法都可用
/opt/ml/code/predict.py
----------
logger = logging.getLogger()
logger.setLevel(logging.INFO)
classpath = <.pkl file>
model = pickle.load(open(classpath, "rb"))
app = flask.Flask(__name__)
print(app)
@app.route("/ping", methods=["GET"]
def ping():
"""Determine if the container is working and healthy."""
return flask.Response(response="Flask running", status=200, mimetype="application/json")
@app.route("/invocations", methods=["POST"])
""InferenceCode""
return flask.Response(response="Invocation Completed", status=200,
mimetype="application/json")
Below snippet was both added and removed , however I still have the endpoint in failed status
if __name__ == '__main__':
app.run(host='0.0.0.0',port=5000)
Error :
"The primary container for production variant <modelname> did not pass the ping health check. Please check CloudWatch logs for this endpoint."
Sagemaker endpoint Cloudwatch logs.
[INFO] Starting gunicorn 20.1.0
[INFO] Listening at: http://0.0.0.0:8000 (1)
[INFO] Using worker: sync
[INFO] Booting worker with pid: 11```
Your predictor file is meant to test if the model is loaded in /ping and if you can perform inference in /invocations.您的预测器文件旨在测试模型是否已加载到 /ping 中,以及您是否可以在 /invocations 中执行推理。 If you have trained your model on SageMaker you need to load it from the /opt/ml directory as follows.
如果您已在 SageMaker 上训练您的模型,您需要按如下方式从 /opt/ml 目录加载它。
prefix = "/opt/ml/"
model_path = os.path.join(prefix, "model")
class ScoringService(object):
model = None # Where we keep the model when it's loaded
@classmethod
def get_model(rgrs):
"""Get the model object for this instance, loading it if it's not already loaded."""
if rgrs.model == None:
with open(os.path.join(model_path, "rf-model.pkl"), "rb") as inp:
rgrs.model = pickle.load(inp)
return rgrs.model
@classmethod
def predict(rgrs, input):
"""For the input, do the predictions and return them.
Args:
input (a pandas dataframe): The data on which to do the predictions. There will be
one prediction per row in the dataframe"""
rf = rgrs.get_model()
return rf.predict(input)
The class helps load your model which we can then verify in the /ping.该类有助于加载您的模型,然后我们可以在 /ping 中进行验证。
# The flask app for serving predictions
app = flask.Flask(__name__)
@app.route("/ping", methods=["GET"])
def ping():
"""Determine if the container is working and healthy. In this sample container, we declare
it healthy if we can load the model successfully."""
health = ScoringService.get_model() is not None # You can insert a health check here
status = 200 if health else 404
return flask.Response(response="\n", status=status, mimetype="application/json")
Here SageMaker will test if you have properly loaded your model.在这里 SageMaker 将测试您是否正确加载了模型。 For /invocations include the inference logic for whatever data format you are passing into your model's predict capabilities.
对于 /invocations 包括您传递到模型预测功能的任何数据格式的推理逻辑。
@app.route("/invocations", methods=["POST"])
def transformation():
data = None
# Convert from CSV to pandas
if flask.request.content_type == "text/csv":
data = flask.request.data.decode("utf-8")
s = io.StringIO(data)
data = pd.read_csv(s, header=None)
else:
return flask.Response(
response="This predictor only supports CSV data", status=415, mimetype="text/plain"
)
print("Invoked with {} records".format(data.shape[0]))
# Do the prediction
predictions = ScoringService.predict(data)
# Convert from numpy back to CSV
out = io.StringIO()
pd.DataFrame({"results": predictions}).to_csv(out, header=False, index=False)
result = out.getvalue()
return flask.Response(response=result, status=200, mimetype="text/csv")
Make sure to setup or configure your predictor.py as shown above so that SageMaker can properly understand/retrieve your model.确保如上所示设置或配置您的 predictor.py,以便 SageMaker 能够正确理解/检索您的模型。
I work for AWS & my opinions are my own.我为 AWS 工作,我的意见是我自己的。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.