Blas SGEMM launch failed

Question

I just installed TensorFlow-GPU 1.0.1 on Win10 GTX GEFORCE 850M with CUDA 8.0 and Cudnn v5.1. when I try to figure out if the installation was successful, I run the

mnist_with_summaries.py

in

C:\\Users...\\Anaconda3\\Lib\\site-packages\\tensorflow\\examples\\tutorials\\mnist

When I run the code in Jupyter Notebook, it prints

Accuracy at step 0: 0.068

Accuracy at step 10: 0.6795

Accuracy at step 10: 0.6795

Accuracy at step 20: 0.8062

Accuracy at step 30: 0.8455

Accuracy at step 40: 0.8737

Accuracy at step 50: 0.8735

Accuracy at step 60: 0.8851

Accuracy at step 70: 0.8815

Accuracy at step 80: 0.8863

Accuracy at step 90: 0.8918

And the kernel just died after print above message.

When I try to run the code in command prompt, it returns error:

failed to create cublas handle

attempting to perform BLAS operation using StreamExecutor without BLAS support

Internal error: Blass SGEMM launch failed: a.shape=(10000,784),b.shape=(784,500)

And this Internal error message appears three times.( too many error message, I just write down something I think useful. If anyone need more information, tell me).

I then try to run:

a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
print(sess.run(c))

And the output is: [[ 22. 28.] [ 49. 64.]] This time the code runs without error. But it should output: Device mapping:

/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 850M

id: 0000:05:00.0

b: /job:localhost/replica:0/task:0/gpu:0

a: /job:localhost/replica:0/task:0/gpu:0

MatMul: /job:localhost/replica:0/task:0/gpu:0

[[ 22. 28.] [ 49. 64.]]

I am totally lost. Could someone tell me why?

Answer 1

How much memory do you have on your graphics card? You may be running out of memory. There are ways to force TensorFlow to limit memory usage-- see: How to prevent tensorflow from allocating the totality of a GPU memory?

But I wonder if TF doesnt handle low memory situations gracefully.