在Flask / Bottle中将tensorflow模型用作REST API

Question

I'm serving my tensorflow model in a Python bottle(or flask) framework but I would like to know how to enable the GPU.

This is how Im creating the session.

with tf.Session(graph=graph) as sess:
        np_pred_confs, np_pred_boxes = sess.run([pred_confs, pred_boxes], feed_dict=feed_dict_testing)

What should be changed to enable the use of the GPU as I'm running the code on a GPU machine.

Answer 1

You need nothing extra to use GPU but just installing GPU version tensorflow. It will automatically use gpu. Further, you need install cuda and cudnn for GPU version tensorflow.

Answer 2

Apart from installing the GPU version of TensorFlow, you also need to install:

1. the necessary GPU drivers - and specifically NVIDIA only
2. CUDA Toolkit 8.0 and
3. cuDNN v6 or v6.1 (select "Download cuDNN v6.0 (April 27, 2017), for CUDA 8.0" )

See the per-OS install notes:

• requirements to run TensorFlow with GPU support, on Ubuntu
• requirements to run TensorFlow with GPU support, on Windows
• TensorFlow with GPU support is not available on Mac

You'll know that it's working correctly if you see this when you run the getting started example]( https://www.tensorflow.org/get_started/get_started#complete_program ):

 Found device 0 with properties: name: GeForce GTX... 

and

 Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX