RuntimeError: cudaGetDevice() 失敗。 狀態：無效參數

Question

我正在嘗試在裝有 Quadro GPU 的筆記本電腦上的 Windows 10 上運行 tensorflow-gpu

-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.21       Driver Version: 465.21       CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro T2000       WDDM  | 00000000:01:00.0  On |                  N/A |
| N/A   59C    P0    14W /  N/A |   2708MiB /  4096MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

當試圖驗證一切是否正常時，我發現 device_lib.list_local_devices() 失敗了

**RuntimeError: cudaGetDevice() failed. Status: invalid argument**


2021-01-13 11:30:14.735823: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: Quadro T2000 computeCapability: 7.5
coreClock: 1.5GHz coreCount: 16 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 104.34GiB/s
2021-01-13 11:30:14.736173: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library
cudart64_110.dll
2021-01-13 11:30:14.736376: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library
cublas64_11.dll
2021-01-13 11:30:14.736590: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library
cublasLt64_11.dll
2021-01-13 11:30:14.736801: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library
cufft64_10.dll
2021-01-13 11:30:14.737016: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library
curand64_10.dll
2021-01-13 11:30:14.737221: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library
cusolver64_10.dll
2021-01-13 11:30:14.737418: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library
cusparse64_11.dll
2021-01-13 11:30:14.737590: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library
cudnn64_8.dll
2021-01-13 11:30:14.737787: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\D041705\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\client\device_lib.py", line
43, in list_local_devices
    _convert(s) for s in _pywrap_device_lib.list_devices(serialized_config)
RuntimeError: cudaGetDevice() failed. Status: invalid argument

任何提示方式發生？ 我有 CUDA 11.2 python 3.8.7 並安裝了 tf 和 tf-gpu 的最新軟件包

Answer 1

剛剛解決了這個問題。 我認為通用解決方案是降級您的 CUDA 和 GPU 驅動程序版本。

---

首先，根據最新一期，TensorFlow 2.4不兼容Cuda 11.2或11.1，改用Cuda 11.0。

其次，如果您使用的是最新的 GPU 驅動程序，您會在運行nvidia-smi時發現 CUDA 版本為 11.3。 將 GPU 驅動程序降級到舊版本，461.09 在我的情況下工作。

---

1. Win10 20H2
2. GTX1070m
3. tensorflow '2.4.1'

英偉達-smi

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 461.09       Driver Version: 461.09       CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+

和

nvcc -V

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:35_Pacific_Daylight_Time_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.relgpu_drvr445TC445_37.28845127_0

要測試 Cuda 是否正常工作，您應該使用 Tensorflow 進行計算，列出設備可能無濟於事。

import tensorflow as tf
tf.debugging.set_log_device_placement(True)

a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
c = tf.matmul(a, b)

output：

Executing op MatMul in device /job:localhost/replica:0/task:0/device:GPU:0