With multiple versions of CUDA installed, how can I make Tensorflow-GPU use a specific version of CUDA on Windows

Question

I currently have two versions of CUDA installed on my computer: 9.0 and 10.0. I have some Python modules that require CUDA 9.0 and some that require 10.0. For example, the version of Tensorflow-GPU I use requires CUDA 10.0. When I try to start training, I get the following error message:

2019-05-23 10:59:35.911847: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2019-05-23 10:59:39.907756: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties:
name: Tesla V100-PCIE-16GB major: 7 minor: 0 memoryClockRate(GHz): 1.38
pciBusID: 0000:84:00.0
totalMemory: 15.90GiB freeMemory: 14.98GiB
2019-05-23 10:59:39.919434: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0
Traceback (most recent call last):
  File "wider_faces_inference.py", line 137, in <module>
    output_dict_array = run_inference_for_images(image_np_list, detection_graph)
  File "wider_faces_inference.py", line 74, in run_inference_for_images
    with tf.Session() as sess:
  File "C:\ProgramData\Anaconda3\envs\tensorflow-gpu\lib\site-packages\tensorflow\python\client\session.py", line 1551, in __init__
    super(Session, self).__init__(target, graph, config=config)
  File "C:\ProgramData\Anaconda3\envs\tensorflow-gpu\lib\site-packages\tensorflow\python\client\session.py", line 676, in __init__
    self._session = tf_session.TF_NewSessionRef(self._graph._c_graph, opts)
tensorflow.python.framework.errors_impl.InternalError: cudaGetDevice() failed. Status: CUDA driver version is insufficient for CUDA runtime version

This I believe is because tensorflow is not looking for the right version of CUDA. I wonder how I can make tensorflow use the correct version of CUDA.

EDIT :

To add a bit more information:

The version of Tensorflow I installed was compiled against CUDA 10.0. I installed CUDA 10.0 and Tensorflow-GPU first, and tensorflow worked just fine. Then I installed CUDA 9.0, and after installation, tensorflow stopped working.

Answer 1

Each version of CUDA comes with a driver that you can choose to install; newer versions of NVidia drivers support older versions of CUDA, but the reverse isn't true. The driver that comes with CUDA 9.0 is not able to run CUDA 10.0 applications.

All that you need to do is install the latest NVidia driver (or generally, any NVidia driver that has been released since CUDA 10.0) in order to have support for CUDA 9.X and 10.0 applications. The path of least resistance might be to reinstall the driver that came with CUDA 10.0, but you should get the most recent driver regardless.