在GRU中使用嵌入层

Question

I have a working code using GRU creating the input manually as a 3D array (None,10,64). The code is:

model = Sequential()
model.add(GRU(300, return_sequences=False, input_shape=(None, 64)))
model.add(Dropout(0.8)) 
model.add(Dense(64, input_dim=300))       
model.add(Activation("linear"))

This returns the predicted embedding given the input window. Now I want to use the keras embedding layer on top of GRU. My idea is to input a 2D array (None, 10) and use the embedding layer to convert each sample to the corresponding embedding vector.

So now I have this:

model = Sequential()
model.add(Embedding(vocab_size, 64, weights=[embedding_matrix], input_length=10, trainable=False))
model.add(GRU(300, return_sequences=False))
model.add(Dropout(0.8)) 
model.add(Dense(64))  
model.add(Activation("linear")) 

I see from the summary that the output of the embedding layer is:

embedding_2 (Embedding)      (None, 10, 64) 

which is what I expected. But when I try to fit the model I get this error:

expected activation_2 to have shape (64,) but got array with shape (1,)

If I comment the other layers and leave only the embedding and gru I get:

expected gru_5 to have shape (300,) but got array with shape (1,)

So my question is what is the difference between fitting a manually constructed 3D array and an embedding layer generated one?.

Answer 1

Your model reflects the desired computation; however, the error is Y you are passing to the model. You are passing a scalar target instead of an array of size (64,) . To clarify your inputs should be sequence of integers, but your targets still need to be vectors.

Also, Dense by default has linear activation, so you don't need the Activation('linear') after Dense(64) .