如何使用 MobileNet 计算图像分类的精度和召回率

Question

I am trying to compute precision, recall, and f1 score on my test dataset. However, I am using ImageDataGenerator format, not using train_test_split (x_train, y_train, x_test and y_test). That's why I couldn't find any references online.

IMAGE_SIZE = 224
BATCH_SIZE = 64

EPOCH = 30
CHANNEL = 3
CLASSES = 10

train_path = "/Users/ba/Documents/mycodes/datasets/DS/train"
valid_path = "/Users/ba/Documents/mycodes/datasets/DS/val"
test_path = "/Users/ba/Documents/mycodes/datasets/DS/test"

train_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.mobilenet_v3.preprocess_input) \
    .flow_from_directory(directory=train_path, target_size=(IMAGE_SIZE,IMAGE_SIZE), batch_size=BATCH_SIZE)
valid_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.mobilenet_v3.preprocess_input) \
    .flow_from_directory(directory=valid_path, target_size=(IMAGE_SIZE,IMAGE_SIZE), batch_size=BATCH_SIZE)
test_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.mobilenet_v3.preprocess_input) \
    .flow_from_directory(directory=test_path, target_size=(IMAGE_SIZE,IMAGE_SIZE), batch_size=BATCH_SIZE, shuffle=False)

Then I tried to calculate precision, recall, and f1 in the following way down below:

from sklearn.metrics import accuracy_score
from sklearn.metrics import precision_score
from sklearn.metrics import recall_score
from sklearn.metrics import f1_score

y_pred_logits = model.predict(test_batches)
y_pred = tf.math.argmax(y_pred_logits)

test_classes = test_batches.classes


# accuracy: (tp + tn) / (p + n)
accuracy = accuracy_score(test_classes, y_pred)
print('Accuracy: %f' % accuracy)
# precision tp / (tp + fp)
precision = precision_score(test_classes, y_pred)
print('Precision: %f' % precision)
# recall: tp / (tp + fn)
recall = recall_score(test_classes, y_pred)
print('Recall: %f' % recall)
# f1: 2 tp / (2 tp + fp + fn)
f1 = f1_score(test_classes, y_pred)
print('F1 score: %f' % f1)

Unfortunately it throws this error message:

ValueError: Found input variables with inconsistent numbers of samples: [1887, 10]

Can you help me re-write the code, or any other references using ImageDataGenerator format I used?

Answer 1

So the problem were on y_true (test_classes) and y_pred. With this, one can also calculate the confusion matrix.

#Making prediction
y_true = test_batches.classes
predictions = model.predict(x=test_batches, steps=len(test_batches), verbose=0)
y_pred = predictions.argmax(axis=1)

print("Precision Score: ",precision_score(y_true, y_pred, pos_label='positive', average='micro'))
print("Recall Score: ",recall_score(y_true, y_pred, pos_label='positive', average='micro'))
print("F1 Score: ",f1_score(y_true, y_pred, pos_label='positive', average='micro'))
print("Accuracy Score: ",accuracy_score(y_true, y_pred))