Tensorflow Keras CuDNN LSTM layers do not support Masking

Question

I am trying to build a NER using TF keras, when I do zero padding to get the batches to same length and then add a tf.keras.layers.Masking() layer to mask the padding, the error I get is that CuDNNLSTM does not support Masking.

My Tensorflow-gpu version is 1.11, Cuda 9.0, CudNN 7.3.1.

I currently do have the version working where I just group same length samples into each batch to avoid padding, I would like to know if we should just not pad from now on or if CuDNN LSTM will support masking paddings in the future?

Thanks

Answer 1

Just found this

CuDNN RNNs cannot be used with masking for the time being, and likely never will be. It would need to be implemented at the CuDNN level.

see here