如何通过docker容器正确安装TFServing

Question

I have tried to install TFServing in docker container several times. However I still can't build it without any error.
I follow the installation steps on the official site. But I still meet compile error during building. I doubt that if there is some flaw in dockerfile which I built. I attach the screenshot of the error.

Answer 1

As of now (Oct 2019), official docker image of TFServing for both CPU & GPU is available at https://hub.docker.com/r/tensorflow/serving .

To set up your tfserving image on docker, just pull the image & start the container.

Pull the official image

docker pull tensorflow/serving:latest

Start the container

docker run -p 8500:8500 -p 8501:8501 --mount type=bind,source=/path/to/model/dir,target=/models/inception --name tfserve -e MODEL_NAME=inception -t tensorflow/serving:latest

For using GPU, pull GPU specific image and pass the appropriate parameters in docker command

docker pull tensorflow/serving:latest-gpu

docker run -p 8500:8500 -p 8501:8501 --mount type=bind,source=/path/to/model/dir,target=/models/inception --name tfserve_gpu -e MODEL_NAME=inception --gpus all -t tensorflow/serving:latest-gpu --per_process_gpu_memory_fraction=0.001

Please note,

gpus all flag is used to allocate all available GPUs ( in case you have multiple in your machine ) to the docker container.

User gpus device=1 to select first GPU device, if you need to restrict usage to a particular device.

per_process_gpu_memory_fraction flag is used to restrict GPU memory usage by the tfserving docker image. Pass its value according to your program need.

Answer 2

I found a dockerfile on the net which meet my demand. https://github.com/posutsai/DeepLearning_Docker/blob/master/Tensorflow-serving/Dockerfile-Tensorflow-serving-gpu