How to run pretrained tensorflow model using Nvidia's Tensor RT on Jetson TX1?

Question

In the Nvidia's blog, they introduced their TensorRT as follows:

NVIDIA TensorRT™ is a high performance neural network inference engine for production deployment of deep learning applications. TensorRT can be used to rapidly optimize, validate and deploy trained neural network for inference to hyperscale data centers, embedded, or automotive product platforms.

So I am wondering, if I have a pre-trained Tensorflow model, can I use it in TensorRT in Jetson TX1 for inference?

Answer 1

UPDATE (2020.01.03): Now both TensorFlow 1.X and 2.0 have been supported by TensorRT (Tested on Trt V6 & 7 : See this tutorial: https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html ).

Base on this post from Nvidia forum, it seems that you could use TensorRT for inference with caffemodel but not tensorflow model now. Beside tensorRT, building tensorflow on tx1 is another issue (refer here: https://github.com/ugv-tracking/cfnet ).

Answer 2

From JetPack 3.1, NVIDIA has added TensorRT support for Tensorflow also. So, the trained TF model can be directly deployed in Jetson TX1/TK1/TX2