What is cuDNN implementation of rnn cells in Tensorflow

Question

To create RNN cells, there are classes like GRUCell and LSTMCell which can be used later to create RNN layers.
And also there are 2 other classes as CudnnGRU and CudnnLSTM which can be directly used to create RNN layers.

In the documentation they say that the latter classes have cuDNN implementation. Why should I use or not use this cuDNN implemented classes over classical RNN implementations when I'm creating a RNN model..?

Answer 1

In short: cudnnGRU and cudnnLSTM can/ must be used on GPU, normal rnn implementations not. So if you have tensorflow-gpu, cudnn implementation of RNN cells would run faster.

Answer 2

CuDNNLSTM and CuDNNGRU are the fast implementation backed by CuDNN . Both can only be run on the GPU , with the TensorFlow backend. The cuDNN is a GPU-accelerated library of primitives for deep neural networks.

The cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, pooling, normalization, and activation layers. cuDNN is part of the NVIDIA Deep Learning SDK.

The cuDNN highlights include:

1. Up to 3x faster training of ResNet-50 and GNMT on Tesla V100 vs. Tesla P100
2. Improved NHWC support for pooling and strided convolution
3. Get Improved performance for common workloads such as ResNet50 and SSD as batchnorm now supports NHWC data layout with an added option to fuse batchnorm with Add and ReLu operations