[英]Tensorflow, Keras: In a multi-class classification, accuracy is high, but precision, recall, and f1-score is zero for most classes
General Explanation: My codes work fine, but the results are wired.一般说明:我的代码工作正常,但结果是连线的。 I don't know the problem is with
我不知道问题出在
I am struggling with this error several weeks and so far I have changed the loss function, optimizer, data generator, etc., but I could not solve it.几个星期以来,我一直在为这个错误而苦苦挣扎,到目前为止我已经更改了损失函数、优化器、数据生成器等,但我无法解决它。 I appreciate any help.
我很感激任何帮助。 If the following information is not enough, let me know, please.
如果以下信息不够,请告诉我。
Field of study: I am using tensorflow, keras for multiclass classification.研究领域:我使用 tensorflow、keras 进行多类分类。 The dataset has 36 binary human attributes.
该数据集具有 36 个二进制人类属性。 I have used resnet50, then for each part of the body (head, upper body, lower body, shoes, accessories), I have added a separated branch to the network.
我用过resnet50,然后对身体的每个部位(头部,上半身,下半身,鞋子,配饰),我都添加了一个单独的分支到网络中。 The network has 1 input image with 36 labels and 36 output nodes (36 denes layers with sigmoid activation).
该网络有 1 个输入图像,具有 36 个标签和 36 个输出节点(36 个具有 sigmoid 激活的定义层)。
Problem: The problem is that the accuracy that keras is reporting is high, but f1-score is very low or zero for most of the outputs (even when I use f1-score as a metric when compiling the network, the f1-socre for validation is very bad).问题:问题是 keras 报告的准确度很高,但是对于大多数输出,f1-score 非常低或为零(即使我在编译网络时使用 f1-score 作为度量,f1-socre 用于验证非常糟糕)。
aAfter train, when I use the network in prediction mode, it returns always one/zero for some classes. a 训练后,当我在预测模式下使用网络时,对于某些类,它总是返回 1/0。 It means that the network is not able to learn (even when I use weighted loss function or focal loss function.)
这意味着网络无法学习(即使我使用加权损失函数或焦点损失函数。)
Why it is weird?为什么奇怪? Because, state-of-the-art methods report heigh f1 score even after the first epoch (eg https://github.com/chufengt/iccv19_attribute , that I have run it in my PC and got good results after one epoch).
因为,即使在第一个 epoch 之后,最先进的方法也报告了很高的 f1 分数(例如https://github.com/chufengt/iccv19_attribute ,我已经在我的 PC 上运行它并在一个 epoch 后得到了很好的结果)。
Parts of the Codes:部分代码:
print("setup model ...")
input_image = KL.Input(args.img_input_shape, name= "input_1")
C1, C2, C3, C4, C5 = resnet_graph(input_image, architecture="resnet50", stage5=False, train_bn=True)
output_layers = merged_model (input_features=C4)
model = Model(inputs=input_image, outputs=output_layers, name='SoftBiometrics_Model')
...
print("model compiling ...")
OPTIM = optimizers.Adadelta(lr=args.learning_rate, rho=0.95)
model.compile(optimizer=OPTIM, loss=binary_focal_loss(alpha=.25, gamma=2), metrics=['acc',get_f1])
plot_model(model, to_file='model.png')
...
img_datagen = ImageDataGenerator(rotation_range=6, width_shift_range=0.03, height_shift_range=0.03, brightness_range=[0.85,1.15], shear_range=0.06, zoom_range=0.09, horizontal_flip=True, preprocessing_function=preprocess_input_resnet, rescale=1/255.)
img_datagen_test = ImageDataGenerator(preprocessing_function=preprocess_input_resnet, rescale=1/255.)
def multiple_outputs(generator, dataframe, batch_size, x_col):
Gen = generator.flow_from_dataframe(dataframe=dataframe,
directory=None,
x_col = x_col,
y_col = args.Categories,
target_size = (args.img_input_shape[0],args.img_input_shape[1]),
class_mode = "multi_output",
classes=None,
batch_size = batch_size,
shuffle = True)
while True:
gnext = Gen.next()
# return image batch and 36 sets of lables
labels = gnext[1]
output_dict = {"{}_output".format(Category): np.array(labels[index]) for index, Category in enumerate(args.Categories)}
yield {'input_1':gnext[0]}, output_dict
trainGen = multiple_outputs (generator = img_datagen, dataframe=Train_df_img, batch_size=args.BATCH_SIZE, x_col="Train_Filenames")
testGen = multiple_outputs (generator = img_datagen_test, dataframe=Test_df_img, batch_size=args.BATCH_SIZE, x_col="Test_Filenames")
STEP_SIZE_TRAIN = len(Train_df_img["Train_Filenames"]) // args.BATCH_SIZE
STEP_SIZE_VALID = len(Test_df_img["Test_Filenames"]) // args.BATCH_SIZE
...
print("Fitting the model to the data ...")
history = model.fit_generator(generator=trainGen,
epochs=args.Number_of_epochs,
steps_per_epoch=STEP_SIZE_TRAIN,
validation_data=testGen,
validation_steps=STEP_SIZE_VALID,
callbacks= [chekpont],
verbose=1)
There is a possibility that you are passing binary f1-score to compile
function.您有可能将二进制 f1-score 传递给
compile
函数。 This should fix the problem -这应该可以解决问题-
pip install tensorflow-addons
...
import tensorflow_addons as tfa
f1 = tfa.metrics.F1Score(36,'micro' or 'macro')
model.compile(...,metrics=[f1])
You can read more about how f1-micro and f1-macro is calculated and which can be useful here .您可以在此处阅读有关如何计算 f1-micro 和 f1-macro 以及哪些可能有用的更多信息。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.