I'm trying to replace the embedding layer in a Keras NLP model. I've trained the model for one language, but I would like to transfer it to another language for which I have comparable embeddings. I hope to achieve this by replacing the index-to-embedding mapping for the source language by the index-to-embedding mapping for the target language.
I've tried to do it like this:
from keras.layers import Embedding
from keras.models import load_model
filename = "my_model.h5"
model = load_model(filename)
new_embedding_layer = Embedding(1000, 300, weights=[my_new_embedding_matrix], trainable=False)
new_embedding_layer.build((None, None))
model.layers[0] = new_embedding_layer
When I print out the model summary, this seems to have worked: the new embedding layer has the correct number of parameters (1000*300=300,000):
_________________________________________________________________
None
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embedding_85 (Embedding) multiple 300000
_________________________________________________________________
lstm_1 (LSTM) (None, 128) 219648
_________________________________________________________________
dense_1 (Dense) (None, 23) 2967
=================================================================
Total params: 522,615
Trainable params: 222,615
Non-trainable params: 300,000
However, when I use the new model to process new examples, nothing seems to have changed: it still accepts input sequences that have values larger than the new vocabulary size of 1000, and returns the same predictions as before.
seq = np.array([10000])
model.predict([seq])
I also notice that the output shape of the new embedding layer is "multiple" rather than (None, None, 300). Maybe this is related to the problem?
Can anyone tell me what I'm missing?
If you Embedding layers have the same shape, then you can simply load your model as you did :
from keras.models import load_model
filename = "my_model.h5"
model = load_model(filename)
Then, rather than building a new embedding layer, you can simply set the weights of the old one :
model.layers[idx_of_your_embedding_layer].set_weights(my_new_embedding_matrix)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.