tf.keras custom metric is giving incorrect results

Question

I have implemented a custom metric in tf.keras for a multi label classification problem.

def multilabel_TP(y_true, y_pred, thres = 0.4):
  return (      
                  tf.math.count_nonzero(
                                         tf.math.logical_and(tf.cast(y_true, tf.bool),
                                         tf.cast(y_pred >= thres, tf.bool))
                                       )
         )

count_zero function produces integer results but while running the model it gives me float values. The custom function gives me correct results when tried outside the scope of the keras model.

 8/33 [======>.......................] - ETA: 27s - loss: 0.4294 - multilabel_TP: **121.6250** 

model.compile(loss = 'binary_crossentropy', metrics = multilabel_TP, optimizer= 'adam')
model.fit(train_sentences, y_train, batch_size= 128, epochs = 20, validation_data= (test_sentences, y_test))

Why is this happenning?

Answer 1

What is presented in the keras progress bar is a running mean of your loss/metrics over batches, since the model is being trained on batches and the weights are changing after each batch. This is why you get a floating point value.

Your metric should also return a floating point value, maybe by taking a division over the number of elements in the batch. Then the metric values will make more sense.