Tensorflow MaxPool doesn't accept float64

Question

I am using Tensorflow to train a CNN. I am currently basing my calculations on Float32 which is kind of default at the time of initialising variables.

I guessed that by using float64 as my dtype I can get more accurate results so I changed the initilization of my varibales as follows:

w1 = tf.Variable(tf.random_normal([5, 5, 3, 64], stddev=0.1, dtype=tf.float64))

But I get following error in the maxpool operation: I chacked the maxpool documentation and it accept value type as follow: value: A 4-D Tensor with shape [batch, height, width, channels] and type float32, float64, qint8, quint8, qint32.

But I get the following error. Is this a bug or I'm doing something wrong?

Input 'input' of 'MaxPool' Op has type float64 that does not match expected type of float32.

---

Traceback (most recent call last):
  File "/Users/hamedketabdar/LearningTensorFlow/CIFAR-Khodam/convolutional_network_batch_2d2c_clean_64f.py", line 213, in <module>
    pred = conv_net(x, weights, biases, keep_prob)
  File "/Users/hamedketabdar/LearningTensorFlow/CIFAR-Khodam/convolutional_network_batch_2d2c_clean_64f.py", line 153, in conv_net
    conv1 = max_pool(conv1, k=2) # Normally K=2
  File "/Users/hamedketabdar/LearningTensorFlow/CIFAR-Khodam/convolutional_network_batch_2d2c_clean_64f.py", line 135, in max_pool
    return tf.nn.max_pool(img, ksize=[1, k, k, 1], strides=[1, k, k, 1], padding='SAME')
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/nn_ops.py", line 235, in max_pool
    name=name)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 449, in _max_pool
    strides=strides, padding=padding, name=name)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/op_def_library.py", line 430, in apply_op
    (prefix, dtypes.as_dtype(input_arg.type).name))
TypeError: Input 'input' of 'MaxPool' Op has type float64 that does not match expected type of float32.

Answer 1

Support for tf.nn.max_pool() for types other than single-precision floating point values is currently not implemented, and the documentation is incorrect. (I have updated it upstream, and it should appear on GitHub and the website soon.)

The reason for the incompatibility is that TensorFlow has a specialized implementation of max-pooling on GPUs for performance reasons, and we almost always use tf.float32 when training deep networks, so there isn't equivalent support for the other types. It would be possible to add, so contributions are welcome: see the GitHub issue for more discussion.