CNN weights on RNN model

Question

是否有可能在 RNN 模型上使用 CNN 权重，就像您可以使用 CNN 的权重来学习每个字母的样子并使用来自该 CNN 的权重来翻译整个句子吗？

Answer 1

It is possible. For an RNN you have to define what is the hidden state so you may set it to be the output of the CNN on the previous element in the sequence. There are several models with state-of-the-art performance on sentence translation. One of them is the Transformer which is makes use of convolutions, as in CNNs, and self-attention. However, it doesn't do so in an RNN scheme, as RNNs are slower to train and evaluate due to their sequential nature. Also note that RNNs are inferior (mainly) due to their problem of short-term memory. You may want to read on LSTMs (that became their successor).