ValueError: Shapes (None, 2) and (None, 1) are incompatible

Question

I'm training a model to identify foreign objects in an image. This is my data generator:

train_data_gen = train_image_generator.flow_from_dataframe(
    traincsv,
    directory=basepath,
    x_col='image_name',
    y_col='class',
    target_size=IMG_SHAPE,
    color_mode='rgb',
    class_mode='binary',
    batch_size=BATCH_SIZE,
    shuffle=True)
    #save_to_dir='/content/drive/My Drive/Results')

validation_data_gen = validation_image_generator.flow_from_dataframe(
    valcsv,
    directory=valpath,
    x_col='image_name',
    y_col='class',
    target_size=IMG_SHAPE,
    color_mode='rgb',
    class_mode='binary',
    batch_size=BATCH_SIZE,
    shuffle=True)
    #save_to_dir='/content/drive/My Drive/Results')

I've loaded Resnet and tried to do transfer learning. This is the model creation:

model = tf.keras.Sequential([
  feature_extractor,
  layers.Dense(2)
])

When I'm compiling with accuracy metrics:

 model.compile( 
  optimizer='adam',
  loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
  metrics=['accuracy'])

and try to fit it:

history = model.fit(train_data_gen,
                    epochs=EPOCHS,
                    validation_data=validation_data_gen)

it runs successfully and gives the accuracy results.

but when I change in Compile the metric to AUC:

 model.compile( 
  optimizer='adam',
  loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
  metrics=['AUC'])

I get an error:

Epoch 1/5

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

<ipython-input-37-909955559916> in <module>()
     13   history = model.fit(train_data_gen,
     14                     epochs=EPOCHS,
---> 15                     validation_data=validation_data_gen)
     16 
     17   t=time.time()

10 frames

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    966           except Exception as e:  # pylint:disable=broad-except
    967             if hasattr(e, "ag_error_metadata"):
--> 968               raise e.ag_error_metadata.to_exception(e)
    969             else:
    970               raise

ValueError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
        outputs = self.distribute_strategy.run(
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
        return fn(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:543 train_step  **
        self.compiled_metrics.update_state(y, y_pred, sample_weight)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/compile_utils.py:411 update_state
        metric_obj.update_state(y_t, y_p)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/metrics_utils.py:90 decorated
        update_op = update_state_fn(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/metrics.py:2083 update_state
        label_weights=label_weights)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/metrics_utils.py:351 update_confusion_matrix_variables
        y_pred.shape.assert_is_compatible_with(y_true.shape)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/tensor_shape.py:1117 assert_is_compatible_with
        raise ValueError("Shapes %s and %s are incompatible" % (self, other))

    ValueError: Shapes (None, 2) and (None, 1) are incompatible

Can someone assists with an idea of how to solve that?

Answer 1

As your problem has just two classes you should use binary crossentropy and the output of your model should be one single neuron:

model = tf.keras.Sequential([
                              feature_extractor,
                              layers.Dense(1)
                             ])
model.compile( 
               optimizer='adam',
               loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
               metrics=['AUC']
             )