[英]How to increase AWS Sagemaker invocation time out while waiting for a response
I deployed a large 3D model to aws sagemaker.我部署了一个大的 3D model 到 aws sagemaker。 Inference will take 2 minutes or more.
推理需要 2 分钟或更长时间。 I get the following error while calling the predictor from Python:
从 Python 调用预测器时出现以下错误:
An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (0) from model with message "Your invocation timed out while waiting for a response from container model. Review the latency metrics for each container in Amazon CloudWatch, resolve the issue, and try again."'
In Cloud Watch I also see some PING time outs while the container is processing:在 Cloud Watch 中,我还看到容器正在处理时出现一些 PING 超时:
2020-10-07T16:02:39.718+02:00 2020/10/07 14:02:39 https://forums.aws.amazon.com/ 106#106: *251 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.32.0.2, server: , request: "GET /ping HTTP/1.1", upstream: "http://unix:/tmp/gunicorn.sock/ping", host: "model.aws.local:8080"
How do I increase the invocation time out?如何增加调用超时?
Or is there a way to make async invocations to an sagemaker endpoint?或者有没有办法对 sagemaker 端点进行异步调用?
It's currently not possible to increase timeout—this is an open issue in GitHub. Looking through the issue and similar questions on SO, it seems like you may be able to use batch transforms in conjunction with inference.目前无法增加超时时间——这是 GitHub 中的一个未决问题。查看 SO 上的问题和类似问题,您似乎可以将批量转换与推理结合使用。
https://stackoverflow.com/a/55642675/806876 https://stackoverflow.com/a/55642675/806876
Sagemaker Python SDK timeout issue: https://github.com/aws/sagemaker-python-sdk/issues/1119 Sagemaker Python SDK 超时问题: https://github.com/aws/sagemaker-python-sdk/issues/1119
This timeout is actually specified at server side - endpoint to be specific.这个超时实际上是在服务器端指定的——具体来说是端点。 You can try the way of
bring your own container
also known as BYOC
, this way you get full control of everything on endpoint side including the timeout.您可以尝试使用
bring your own container
(也称为BYOC
)的方式,这样您就可以完全控制端点端的所有内容,包括超时。
You can also reference the endpoint part of this repo which is from one of my colleague - https://github.com/jackie930/yolov4-SageMaker您还可以参考此回购协议的端点部分,它来自我的一位同事 - https://github.com/jackie930/yolov4-SageMaker
The timeout you should change exists in serve.py
model_server_timeout = os.environ.get('MODEL_SERVER_TIMEOUT', 60)
您应该更改的超时存在于
serve.py
model_server_timeout = os.environ.get('MODEL_SERVER_TIMEOUT', 60)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.