Triton 推理服务器 - tritonserver：未找到

Question

I try to run NVIDIA's Triton Inference Server. I pulled the pre-built container nvcr.io/nvidia/pytorch:22.06-py3 and then run it with the command

run --gpus=1 --rm -p8000:8000 -p8001:8001 -p8002:8002 -v/F/models:/models nvcr.io/nvidia/pytorch:22.06-py3 tritonserver --model-repository=/models

and got the error

/opt/nvidia/nvidia_entrypoint.sh: line 49: exec: tritonserver: not found

I googled and have not found something to catch this. I tried to change tritonserver to trtserver as recommended but it did not help. Please give some advice how it can be solved.

Answer 1

Looks like you're trying to run a tritonserver using a pytorch image but according to the triton-server quick start guide , the image should be:

$ docker run --gpus=1 --rm -p8000:8000 -p8001:8001 -p8002:8002 -v/full/path/to/docs/examples/model_repository:/models nvcr.io/nvidia/tritonserver:<xx.yy>-py3 tritonserver --model-repository=/models

Where <xx.yy> is the version of Triton that you want to use

In your case it should be nvcr.io/nvidia/tritonserver:22.06-py3 and the full command:

run --gpus=1 --rm -p8000:8000 -p8001:8001 -p8002:8002 -v/F/models:/models nvcr.io/nvidia/tritonserver:22.06-py3 tritonserver --model-repository=/models