Dimension error with RNN and word classification

Question

I'm pretty new with NLP and I want to classify different words depending on their language (basically my model should tell me if a word is french, or english, or spanish and so on).

When I fit the following model I get a dimension error. The "dataset" contains the words, it's a padded tensor of size (1550, 19) and the "y" contains the different languages, it's also a padded tensor of size (1550, 10).

np.random.seed(42)
tf.random.set_seed(42)

from tensorflow.keras.layers import LSTM, GRU, Input, Embedding, Dense

input = Input(shape=[None])
z = Embedding(max_id + 1, 128, input_shape=[None], mask_zero=True)(input)
z = GRU(128)(z)
output = Dense(18, activation='softmax')(z)

model = keras.models.Model(input, output)
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

h = model.fit(dataset, y, epochs=5)

ValueError: Shapes (None, 10) and (None, 18) are incompatible

Do you see where the problem is?

Thanks!

Answer 1

The message tells you that the shapes are not compatible, they need to match. I would have put this as a comment, but I can't due to my reputation, so that's why I answered directly, however I am not sure if it works, have you tried:

output = Dense(10, activation='softmax')(z)