Tensorflow dimension issue: ValueError: Shapes (3, 1) and (None, 3) are incompatible

Question

I'm pretty new with NN and I am having an issue with some dimensions whilst fitting a model. Here's my case:

model_sigmoid = tf.keras.Sequential([
  embedding_layer,
  GlobalAveragePooling1D(),
  Dense(3, activation="softmax")])

model_sigmoid.summary()

Model: "sequential_12"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding (Embedding)        (None, None, 100)         1195200   
_________________________________________________________________
global_average_pooling1d_5 ( (None, 100)               0         
_________________________________________________________________
dense_11 (Dense)             (None, 3)                 303       
=================================================================
Total params: 1,195,503
Trainable params: 303
Non-trainable params: 1,195,200
___________________________________________

This is the model I would like to train (it's a model to set a starting baseline). It's a multiclass classification problem with an embedding layer: GloVe 100d embedding

model_sigmoid.compile(optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"])

history = model_sigmoid.fit(
        train, epochs=10, batch_size=128, 
        validation_data=validation, verbose=1
    )

train and validation are vectorized version of my train and validation dataset.

train_ds
<MapDataset shapes: ((None, 80), (3,)), types: (tf.int64, tf.float32)>
tweet, label = next(iter(train))

tweet
<tf.Tensor: shape=(1, 80), dtype=int64, numpy=
array([[   6,   32, 1321,    3,  157,  383,    4,   18,  137, 1222,    6,
          18,  181, 2770, 1024, 6781,   51,    6,  375,  240,  486,    0,
           0,    0,    0,    0,    0,    0,    0,    0,    0,    0,    0,
           0,    0,    0,    0,    0,    0,    0,    0,    0,    0,    0,
           0,    0,    0,    0,    0,    0,    0,    0,    0,    0,    0,
           0,    0,    0,    0,    0,    0,    0,    0,    0,    0,    0,
           0,    0,    0,    0,    0,    0,    0,    0,    0,    0,    0,
           0,    0,    0]])>

label
<tf.Tensor: shape=(3,), dtype=float32, numpy=array([1., 0., 0.], dtype=float32)>

As you can see the my "X" is a sequence with a length of 80 and with integers that correspond to the initial words in my dataset. My "Y" instead is an encoded version of the original sentiment value (negative, neutral, positive).

When I call the fit operation I get

ValueError: Shapes (3, 1) and (None, 3) are incompatible

I'm pretty sure the error is with Y, but I can't really figure out how to fix the shape of my tensor.

Answer 1

After some digging and more shape inspection, I figured out how to fix the error above.

I added a reshape call in my function:

def vectorize_text_and_reshape(text, label):
      text = tf.expand_dims(text, -1)
      return vectorizer(text), tf.reshape(label, [1,3]) 

train_ds = train_tf.map(vectorize_text_and_reshape)
val_ds = val_tf.map(vectorize_text_and_reshape)
test_ds = test_tf.map(vectorize_text_and_reshape)

I had already implemented the vectorize_text_and_reshape function above for vectorizing my text data. I only had to add the reshape call at the label level. This turned my label from (3,) shape to (1,3).