ValueError：形状（无，无）和（无，无，无，3）不兼容

Question

I am using EfficientNetV2B0, everything works perfectly until when I try to fit the model. On the train_generator I tried the class_mode = 'sparse', and also class_mode = 'categorical', it throws different kinds of error messages. It took me few days, but I couldn't solve it out, can someone please help?

Here the full error message:

ValueError: in user code:

File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/engine/training.py", line 1051, in train_function  *
    return step_function(self, iterator)
File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/engine/training.py", line 1040, in step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/engine/training.py", line 1030, in run_step  **
    outputs = model.train_step(data)
File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/engine/training.py", line 890, in train_step
    loss = self.compute_loss(x, y, y_pred, sample_weight)
File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/engine/training.py", line 948, in compute_loss
    return self.compiled_loss(
File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/engine/compile_utils.py", line 201, in __call__
    loss_value = loss_obj(y_t, y_p, sample_weight=sw)
File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/losses.py", line 139, in __call__
    losses = call_fn(y_true, y_pred)
File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/losses.py", line 243, in call  **
    return ag_fn(y_true, y_pred, **self._fn_kwargs)
File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/losses.py", line 1787, in categorical_crossentropy
    return backend.categorical_crossentropy(
File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/backend.py", line 5119, in categorical_crossentropy
    target.shape.assert_is_compatible_with(output.shape)

ValueError: Shapes (None, None) and (None, None, None, 3) are incompatible

Here is my code:

x = base_model.layers[-6].output
x = Dense(1024,activation='relu')(x) #dense layer 1
x = Dense(512,activation='relu')(x) #dense layer 2
output = Dense(CLASSES, activation='softmax')(x) #final layer with softmax activation
model = Model(inputs=base_model.input, outputs=output)

I tried both CategoricalCrossentropy and SparseCategoricalCrossentropy

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001), 
              loss=tf.keras.losses.CategoricalCrossentropy(), 
              metrics=['accuracy'])

history = model.fit(
    x=train_generator,
    steps_per_epoch = 54,
    epochs = EPOCH,
    validation_data = validation_generator,
    validation_steps = 6,
    verbose = 2,
    shuffle = True
)

When I try SparseCategoricalCrossentropy()

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001), 
                  loss=tf.keras.losses.SparseCategoricalCrossentropy(), 
                  metrics=['accuracy'])

It throws a different error message:

InvalidArgumentError: Graph execution error:

File "/Users/ba/opt/anaconda3/envs/tensorflow/lib/python3.8/site-packages/keras/backend.py", line 5238, in sparse_categorical_crossentropy
      res = tf.nn.sparse_softmax_cross_entropy_with_logits(
Node: 'sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits'
logits and labels must have the same first dimension, got logits shape [1568,3] and labels shape [32]
     [[{{node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits}}]] [Op:__inference_train_function_26092]

Answer 1

The problem here is that you are feeding the loss function with two different structures: one is a 2 dimensional tensor (None, None) the other is a 4 dimensional tensor (None, None, None, 3)

You will need to check how the couple <data, target> is, and make sure to feed the loss with two tensors having the same shape. This is general for all the losses you are gonna use.

Answer 2

The 2D Global average pooling block takes a tensor of size (input width) x (input height) x (input channels) and computes the average value of all values across the entire (input width) x (input height) matrix for each of the (input channels).

Thus this code solved everything: x = GlobalAveragePooling2D()(x)

x = base_model.layers[-6].output
x = GlobalAveragePooling2D()(x)
x = Dense(1024,activation='relu')(x) #dense layer 1
x = Dense(512,activation='relu')(x) #dense layer 2
output = Dense(CLASSES, activation='softmax')(x) #final layer with softmax activation
model = Model(inputs=base_model.input, outputs=output)