在预训练的Keras模型中替换嵌入层

Question

I'm trying to replace the embedding layer in a Keras NLP model. I've trained the model for one language, but I would like to transfer it to another language for which I have comparable embeddings. I hope to achieve this by replacing the index-to-embedding mapping for the source language by the index-to-embedding mapping for the target language.

I've tried to do it like this:

from keras.layers import Embedding
from keras.models import load_model

filename = "my_model.h5"
model = load_model(filename)

new_embedding_layer = Embedding(1000, 300, weights=[my_new_embedding_matrix], trainable=False)
new_embedding_layer.build((None, None))
model.layers[0] = new_embedding_layer

When I print out the model summary, this seems to have worked: the new embedding layer has the correct number of parameters (1000*300=300,000):

_________________________________________________________________
None
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_85 (Embedding)     multiple                  300000    
_________________________________________________________________
lstm_1 (LSTM)                (None, 128)               219648    
_________________________________________________________________
dense_1 (Dense)              (None, 23)                2967      
=================================================================
Total params: 522,615
Trainable params: 222,615
Non-trainable params: 300,000

However, when I use the new model to process new examples, nothing seems to have changed: it still accepts input sequences that have values larger than the new vocabulary size of 1000, and returns the same predictions as before.

seq = np.array([10000])
model.predict([seq])

I also notice that the output shape of the new embedding layer is "multiple" rather than (None, None, 300). Maybe this is related to the problem?

Can anyone tell me what I'm missing?

Answer 1

If you Embedding layers have the same shape, then you can simply load your model as you did :

from keras.models import load_model

filename = "my_model.h5"
model = load_model(filename)

Then, rather than building a new embedding layer, you can simply set the weights of the old one :

model.layers[idx_of_your_embedding_layer].set_weights(my_new_embedding_matrix)