[英]Serving multiple tensorflow models using docker
Having seen this github issue and this stackoverflow post I had hoped this would simply work.看过这个github 问题和这个stackoverflow 帖子后,我希望这能简单地工作。
It seems as though passing in the environment variable MODEL_CONFIG_FILE
has no affect.似乎传入环境变量MODEL_CONFIG_FILE
没有任何影响。 I am running this through docker-compose
but I get the same issue using docker-run
.我正在通过docker-compose
运行它,但我使用docker-run
遇到了同样的问题。
The error:错误:
I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: model model_base_path: /models/model
I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: model
E tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:369] FileSystemStoragePathSource encountered a file-system access error: Could not find base path /models/model for servable model
The Dockerfile Dockerfile
FROM tensorflow/serving:nightly
COPY ./models/first/ /models/first
COPY ./models/second/ /models/second
COPY ./config.conf /config/config.conf
ENV MODEL_CONFIG_FILE=/config/config.conf
The compose file撰写文件
version: '3'
services:
serving:
build: .
image: testing-models
container_name: tf
The config file配置文件
model_config_list: {
config: {
name: "first",
base_path: "/models/first",
model_platform: "tensorflow",
model_version_policy: {
all: {}
}
},
config: {
name: "second",
base_path: "/models/second",
model_platform: "tensorflow",
model_version_policy: {
all: {}
}
}
}
I ran into this double slash issue for git bash on windows.我在 Windows 上遇到了 git bash这个双斜杠问题。
As such I am passing the argument, mentioned by @KrisR89, in via command
in the docker-compose
.因此,我通过 docker docker-compose
中的 via command
传递@KrisR89 提到的参数。
The new docker-compose
looks like this and works with the supplied dockerfile
:新的docker-compose
看起来像这样,并与提供的dockerfile
:
version: '3'
services:
serving:
build: .
image: testing-models
container_name: tf
command: --model_config_file=/config/config.conf
There is no docker environment variable named “MODEL_CONFIG_FILE” (that's a tensorflow/serving variable, see docker image link ), so the docker image will only use the default docker environment variables ("MODEL_NAME=model" and "MODEL_BASE_PATH=/models"), and run the model “/models/model” at startup of the docker image.没有名为“MODEL_CONFIG_FILE”的 docker 环境变量(这是一个 tensorflow/serving 变量,请参阅 docker 镜像链接),因此 docker 镜像将仅使用默认的 docker 环境变量(“MODEL_NAME=model”和“MODEL_BASE_PATH=/models”) ,并在 docker 镜像启动时运行模型“/models/model”。 "config.conf" should be used as input at "tensorflow/serving" startup. “config.conf”应该在“tensorflow/serving”启动时用作输入。 Try to run something like this instead:尝试运行这样的东西:
docker run -p 8500:8500 8501:8501 \
--mount type=bind,source=/path/to/models/first/,target=/models/first \
--mount type=bind,source=/path/to/models/second/,target=/models/second \
--mount type=bind,source=/path/to/config/config.conf,target=/config/config.conf\
-t tensorflow/serving --model_config_file=/config/config.conf
The error is cause serving couldn't find your model.错误是因为服务找不到您的模型。
E tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:369] FileSystemStoragePathSource encountered a file-system access error: Could not find base path /models/model for servable model
Your docker compose file didn't mount your model files in the container.您的 docker compose 文件没有在容器中装载您的模型文件。 So the Serving couldn't find your models.因此 Serving 无法找到您的模型。 I suggest to set three configure files.我建议设置三个配置文件。
1 docker-compose.yml 1 docker-compose.yml
2 .env 2 .env
3 models.config 3个models.config
docker-compose.yml
: docker-compose.yml
:
Mount your model files from host to the container.将模型文件从主机挂载到容器。 I think you could do this :我认为你可以这样做:
version: "3"
services:
sv:
image: tensorflow/serving:latest
restart: unless-stopped
ports:
- 8500:8500
- 8501:8501
volumes:
- ${MODEL1_PATH}:/models/${MODEL1_NAME}
- ${MODEL2_PATH}:/models/${MODEL2_NAME}
- /home/deploy/dcp-file/tf_serving/models.config:/models/models.config
command: --model_config_file=/models/models.config
.env
: docker-compose.yml
loads info from this file. .env
: .env
docker-compose.yml
从此文件加载信息。
MODEL1_PATH=/home/notebooks/water_model
MODEL1_NAME=water_model
MODEL2_PATH=/home/notebooks/ice_model
MODEL2_NAME=ice_model
models.config
: models.config
:
model_config_list: {
config {
name: "water_model",
base_path: "/models/water_model",
model_platform: "tensorflow",
model_version_policy: {
versions: 1588723537
versions: 1588734567
}
},
config {
name: "ice_model",
base_path: "/models/ice_model",
model_platform: "tensorflow",
model_version_policy: {
versions: 1588799999
versions: 1588788888
}
}
}
And you can see this serving official document你可以看到这个服务的官方文档
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.