在theano中使用CUDA8

Question

我已經安裝了CUDA8並安裝了theano，在導入theano時它會搜索CUDA7.5而不是CUDA8，怎么能告訴theano使用CUDA8代替CUDA7.5？

我的系統只有CUDA8，它不包含混合環境cuda（即同時具有CUDA7.5和CUDA8）。

這是nvidia-smi的輸出

$ nvidia-smi 
Sat Feb  4 11:32:30 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.26                 Driver Version: 375.26                         |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 970M    Off  | 0000:01:00.0     Off |                  N/A |
| N/A   54C    P0    22W /  N/A |      0MiB /  3016MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|  No running processes found                                                 |

這是nvcc -V的輸出

$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Sun_Sep__4_22:14:01_CDT_2016
Cuda compilation tools, release 8.0, V8.0.44

在ipython中導入theano時，它無法在gpu模式下運行而出現錯誤，它無法找到libcudart.so.7.5

Python 3.6.0 (default, Jan 16 2017, 12:12:55) 
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import theano
ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: libcudart.so.7.5: cannot open shared object file: No such file or directory
WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu0 is not available  (error: cuda unavailable)

這是我的.theanorc的內容

[global]                                                                                                                                                                                                       
floatX = float32
device = gpu0
cuda.root = /opt/cuda

我嘗試從源代碼構建theano ，在卸載之前的安裝后，它也無法正常工作。 我確實用theano-cache clean / theano-cache purge ，我手動刪除了.theano目錄下的內容，這也.theano 。

隨着更多調試我得到錯誤https://github.com/Theano/Theano/blob/8b9f73365e4932f1c005a0a37b907d28985fbc5f/theano/gof/cmodule.py#L302

當nvcc_compiler嘗試加載cuda_ndarray.so從cuda_ndarray在theano緩存

mod.cu的編譯階段運行沒有錯誤。

在這種情況下，鏈接器指向錯誤的libcudart

readelf -a cuda_ndarray.so | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libcublas.so.8.0]
 0x0000000000000001 (NEEDED)             Shared library: [libpython3.6m.so.1.0]
 0x0000000000000001 (NEEDED)             Shared library: [libcudart.so.7.5]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

我假設ldconfig正在緩存cuda庫

$ sudo ldconfig -v | grep -e 'cuda\|blas'
/opt/cuda/lib64:
    libcublas.so.8.0 -> libcublas.so.8.0.45
    libcudart.so.8.0 -> libcudart.so.8.0.44
    libnvblas.so.8.0 -> libnvblas.so.8.0.44
/opt/cuda/nvvm/lib64:
    libcuda.so.1 -> libcuda.so.375.26
    libblas.so.3 -> libblas.so.3.7.0
    libicudata.so.58 -> libicudata.so.58.2
    libopenblas.so.0 -> libopenblas.so
    libicudata.so.58 -> libicudata.so.58.1

Answer 1

在進一步挖掘我的問題之后，我重構了我原來的問題並在這里發布了nvcc正在挑選錯誤的libcudart庫來解決我的問題。