[英]Deployment with customer handler on Google Cloud Vertex AI
I'm trying to deploy a TorchServe instance on Google Vertex AI platform but as per their documentation ( https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#response_requirements ), it requires the responses to be of the following shape:我正在尝试在 Google Vertex AI 平台上部署一个 TorchServe 实例,但根据他们的文档( https://cloud.google.com/vertex-ai/docs/predictions/custom-container-requirements#response_requirements ),它需要响应具有以下形状:
{
"predictions": PREDICTIONS
}
Where PREDICTIONS is an array of JSON values representing the predictions that your container has generated.其中PREDICTIONS是一个包含 JSON 个值的数组,表示您的容器生成的预测。
Unfortunately, when I try to return such a shape in the postprocess()
method of my custom handler, as such:不幸的是,当我尝试在自定义处理程序的
postprocess()
方法中返回这样的形状时,如下所示:
def postprocess(self, data):
return {
"predictions": data
}
TorchServe returns: TorchServe 返回:
{
"code": 503,
"type": "InternalServerException",
"message": "Invalid model predict output"
}
Please note that data
is a list of lists, for example: [[1, 2, 1], [2, 3, 3]].请注意,
data
是列表的列表,例如:[[1, 2, 1], [2, 3, 3]]。 (Basically, I am generating embeddings from sentences) (基本上,我是从句子生成嵌入)
Now if I simply return data
(and not a Python dictionary), it works with TorchServe but when I deploy the container on Vertex AI, it returns the following error: ModelNotFoundException
.现在,如果我只是返回
data
(而不是 Python 字典),它可以与 TorchServe 一起使用,但是当我在 Vertex AI 上部署容器时,它会返回以下错误: ModelNotFoundException
。 I assumed Vertex AI throws this error since the return shape does not match what's expected (cf documentation).我假设 Vertex AI 抛出此错误,因为返回形状与预期不匹配(参见文档)。
Did anybody successfully manage to deploy a TorchServe instance with custom handler on Vertex AI?有人成功地在 Vertex AI 上部署了带有自定义处理程序的 TorchServe 实例吗?
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.