TX2 GPU无法正确分配内存

Question

I am trying to run a code on TX2 but the tensorflow code that allocates GPU memory usage seems to be working in a weird manner.

Here's the code I have to allocate memory:

config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.0
config.gpu_options.visible_device_list = "0"
set_session(tf.Session(config=config))

The weird thing is, when I use 0.0 instead of 0.5 , the processing is faster. And when I use 0.9 , I get the following error:

tensorflow.python.framework.errors_impl.InternalError: GPU sync failed

What's happening here?

Answer 1

First thing to check will be to verify if compatible CUDA, cuDNN versions are correctly installed and to reboot the system.
Then, Allowing GPU memory growth can help. https://www.tensorflow.org/guide/using_gpu#allowing_gpu_memory_growth
Perhaps, you can try:

import tensorflow as tf       
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config) 
set_session(sess)