在 tensorflow.keras 中使用 sklearn 宏 f1-score 作为度量

Question

我已经为 tensorflow.keras 定义了自定义指标，以便在每个 epoch 之后计算 macro-f1-score，如下所示：

from tensorflow import argmax as tf_argmax
from sklearn.metric import f1_score

def macro_f1(y_true, y_pred):
    # labels are one-hot encoded. so, need to convert
    # [1,0,0] to 0 and
    # [0,1,0] to 1 and
    # [0,0,1] to 2. Then pass these arrays to sklearn f1_score.
    y_true = tf_argmax(y_true, axis=1)
    y_pred = tf_argmax(y_pred, axis=1)
    return f1_score(y_true, y_pred, average='macro')

并在模型编译期间使用它

model_4.compile(loss = 'categorical_crossentropy',
                optimizer = Adam(lr=init_lr, decay=init_lr / num_epochs),
                metrics = [Recall(name='recall') #, weighted_f1
                           macro_f1])

当我尝试像这样适应时：

history_model_4 = model_4.fit(train_image_generator.flow(x=train_imgs, y=train_targets, batch_size=batch_size),
                            validation_data = (val_imgs, val_targets),
                            epochs=num_epochs,
                            class_weight=mask_weights_train,
                            callbacks=[model_save_cb, early_stop_cb, epoch_times_cb],
                            verbose=2)

这是错误：

OperatorNotAllowedInGraphError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:806 train_function  *
        return step_function(self, iterator)
    <ipython-input-57-a890ea61878e>:6 macro_f1  *
        return f1_score(y_true, y_pred, average='macro')
    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1095 f1_score  *
        return fbeta_score(y_true, y_pred, 1, labels=labels,
    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1217 fbeta_score  *
        _, _, f, _ = precision_recall_fscore_support(y_true, y_pred,
    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1478 precision_recall_fscore_support  *
        labels = _check_set_wise_labels(y_true, y_pred, average, labels,
    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:1301 _check_set_wise_labels  *
        y_type, y_true, y_pred = _check_targets(y_true, y_pred)
    /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:80 _check_targets  *
        check_consistent_length(y_true, y_pred)
    /usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py:209 check_consistent_length  *
        uniques = np.unique(lengths)
    <__array_function__ internals>:6 unique  **
        
    /usr/local/lib/python3.6/dist-packages/numpy/lib/arraysetops.py:263 unique
        ret = _unique1d(ar, return_index, return_inverse, return_counts)
    /usr/local/lib/python3.6/dist-packages/numpy/lib/arraysetops.py:311 _unique1d
        ar.sort()
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:877 __bool__
        self._disallow_bool_casting()
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:487 _disallow_bool_casting
        "using a `tf.Tensor` as a Python `bool`")
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:474 _disallow_when_autograph_enabled
        " indicate you are trying to use an unsupported feature.".format(task))

    OperatorNotAllowedInGraphError: using a `tf.Tensor` as a Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

是什么导致了此类错误，我该如何修复它并将其用作每个 epoch 结束时的评估指标之一？

编辑 1：
注意：所有这些都是在 jupyter notebook 中完成的，我添加了“>>>”来分隔线

# getting a batch to pass to model
>>> a_batch = train_image_generator.flow(x=train_imgs, y=train_targets, batch_size=batch_size).next()
# checking its' type to ensure that it's what i though it is
>>> type(a_batch)
# passing the batch to the model
>>> logits = model_4(a_batch)
# checking the type of output
>>> type(logits)
tensorflow.python.framework.ops.EagerTensor
# extracting only the passed targets to calculate f1-score
>>> _, dummy_targets = a_batch
# checking it's type
>>> type(dummy_targets)
numpy.ndarray
>>> macro_f1(y_true=dummy_targets, y_pred=logits)
0.0811965811965812

Answer 1

sklearn不是 TensorFlow 代码 - 始终建议避免在 TF 中使用在 TF 执行图中执行的任意 Python 代码。

TensorFlow 插件已经实现了 F1 分数 ( tfa.metrics.F1Score )，因此请更改您的代码以使用它而不是您的自定义指标

确保你先pip install tensorflow-addons然后

import tensorflow_addons as tfa

model_4.compile(loss = 'categorical_crossentropy',
                optimizer = Adam(lr=init_lr, decay=init_lr / num_epochs),
                metrics = [Recall(name='recall') #, weighted_f1
                           tfa.metrics.F1Score(average='macro')])