如何将RNN与CNN结合？

Question

I'm trying to combine LSTM with CNN but I got stuck because of an error. Here is the model I'm trying to implement:

model=Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(28, 28,3), activation='relu'))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(32, activation='relu'))
model.add(LSTM(128, return_sequences=True,input_shape=(1,32), activation='relu'))
model.add(LSTM(256))
model.add(Dropout(0.25))
model.add(Dense(37))
model.compile(loss='categorical_crossentropy', optimizer='adam')

and error happens in the first LSTM layer:

ERROR: Input 0 is incompatible with layer lstm_12: expected ndim=3, found ndim=2

Answer 1

The input of LSTM layer should be a 3D array which represents a sequence or a timeseries (this is what the error is trying to say: expected ndim=3 ). However, in your model the input of LSTM layer, which is actually the output of the Dense layer before it, is a 2D array (ie found ndim=2 ). To make it into a 3D array of shape (n_samples, n_timesteps, n_features) , one solution is to use a RepeatVector layer to repeat it as much as the number of timesteps (which you need to specify in your code):

model.add(Dense(32, activation='relu'))
model.add(RepeatVector(n_timesteps))
model.add(LSTM(128, return_sequences=True, input_shape=(n_timesteps,32), activation='relu'))