[英]GCP Vertex AI Endpoint returning empty prediction array
KFP pipeline job executes successfully, but upon hitting the endpoint, am getting an empty predictions array ([]). KFP 管道作业成功执行,但在到达端点时,得到一个空的预测数组 ([])。 I suspect the issue is in the model upload, where the model is not registered correctly somehow.
我怀疑问题出在 model 上传中,其中 model 没有以某种方式正确注册。 Any tips are appreciated.
任何提示表示赞赏。
Code to upload model deploy task:上传 model 部署任务的代码:
#Import a model programmatically
model_upload = aiplatform.Model.upload(
display_name = DISPLAY_NAME,
serving_container_image_uri = serving_container_image_uri,
serving_container_health_route="/health_check",
serving_container_predict_route="/predict",
serving_container_ports=[8080],
serving_container_environment_variables={
"MODEL_NAME": MODEL_NAME,
},
)
Code to get predictions:获取预测的代码:
response = endpoint.predict({"user_id": 150})
# response = endpoint.predict({"instances":{"user_id": 150}})
response
Response:回复:
Prediction(predictions=[], deployed_model_id='4656867150235959296', explanations=None)
TLDR: TLDR:
Check your handler output is returning a correctly formatted string without extra slashes ( \
) in the response which meets the requirements .检查您的处理程序 output 是否在满足要求的响应中返回格式正确且没有多余斜杠 (
\
) 的字符串。
Or use raw predict
instead of predict
call.或者使用
raw predict
而不是predict
调用。 ( guide ) ( 指导)
Debugging steps:调试步骤:
I also had this issue where I was using the Vertex AI custom serving container approach and calling endpoint.predict(instances=[{...}]).predictions
was only returning an empty list: []
.我也有这个问题,我使用 Vertex AI 自定义服务容器方法并调用
endpoint.predict(instances=[{...}]).predictions
只返回一个空列表: []
。
Even though, when using raw predict
from the Python SDK or via REST endpoint , it returned a valid response for the data
JSON key: Even though, when using
raw predict
from the Python SDK or via REST endpoint , it returned a valid response for the data
JSON key:
Python SDK response: Python SDK 响应:
"{\"predictions\": \"[{\\\"key1\\\": \\\"string1\\\", \\\"key2\\\": [\\\"string2\\\"], \\\"key3\\\": 0.0}]\"}"
REST API response: REST API 响应:
{"predictions": "[{\"key1\": \"string1\", \"key2\": [\"string2\"], \"key3\": 0.0}]"}
However, in both cases you can see that there were extra slashes ( \
) in the responses.但是,在这两种情况下,您都可以看到响应中有额外的斜杠 (
\
)。 This means that it was a formatting issue .这意味着这是一个格式问题。
Solution:解决方案:
It turned out that in the post-processing step of my handler, I had done the following:事实证明,在我的处理程序的后处理步骤中,我做了以下事情:
prediction_result = {
"key1": value1,
"key2": value2,
"key3": value3,
}
array = np.array([prediction_result])
return json.dumps({"predictions": array.tolist()})
Changing it to the following fixed the problem for me:将其更改为以下解决了我的问题:
prediction_result = {
"key1": value1,
"key2": value2,
"key3": value3,
}
return json.dumps({"predictions": [prediction_result]})
And then calling endpoint.predict(instances=[{...}]).predictions
returned the following list for me:然后调用
endpoint.predict(instances=[{...}]).predictions
为我返回以下列表:
[{'key1': 'string1',
'key2': ['string2'],
'key3': 0.0}]
After the fix, the responses from raw predict
no longer contained the extra slashes ( \
).修复后,来自
raw predict
的响应不再包含额外的斜杠 ( \
)。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.