Getting a ValueError in tensorflow 2.x: ValueError: Shapes (50, 6) and (50, 100) are incompatible

Question

I am fairly new to tensorflow and I have this simple model:

model = keras.Sequential()
model.add(layers.Embedding(input_dim=64, output_dim=32))
model.add(layers.GRU(128, return_sequences=True))
model.add(layers.SimpleRNN(32))
model.add(layers.Dense(100, activation='softmax'))

model.compile(
    optimizer='adam',
    loss=keras.losses.categorical_crossentropy,
    metrics=['accuracy']
)

---

When I try to fit the model with:

model.fit(dataset, epochs=50)

I get the following error: ValueError: Shapes (50, 6) and (50, 100) are incompatible

I have this dataset: <MapDataset shapes: ((50, 6), (50, 6)), types: (tf.int64, tf.int64)>

I used this code to make the dataset:

results = tf.data.Dataset.from_tensor_slices(dataset)
sequences = results.batch(51, drop_remainder=True)
def split(batch):       
    input_ = batch[:-1]
    output_ = batch[1:]
    return input_, output_
dataset = sequences.map(split)

I am trying to make a model that, given a sequence of arrays, it will predict the next element in the sequence. I am using google colab to run my code. Any help will be appreciated.

Answer 1

Looking at your dataset, you 6 classes and 50 samples . Therefore your label shape is (50,6) .

But the output layer of your model has 100 units which essentially means that you have 100 classes, which is not the case. Therefore you get the shape error.

The number of units in the final layer should be equal to the number of classes for softmax activation with categorical_crossentropy loss.

Change the model architecture to this:

model = keras.Sequential()
model.add(layers.Embedding(input_dim=64, output_dim=32))
model.add(layers.GRU(128, return_sequences=True))
model.add(layers.SimpleRNN(32))
model.add(layers.Dense(6, activation='softmax'))