Keras评估生成器准确率高，但每个类的准确率低

Question

I am using Keras ImageDataGenerator to load and train a model on 101-food-dataset. I am using the same split for validation and test . I also use sklearn.metrics.confusion_matrix to calculate accuracy for each class.

After training, when I use evaluate_generator , I get high accuracy. But the accuracy of individual classes is very low. What mistake am I making?

train_generator = train_datagen.flow_from_directory(
  train_data_dir,
  target_size=(img_height, img_width),
  batch_size=batch_size,
  class_mode='categorical',
  subset='training')

validation_generator = train_datagen.flow_from_directory(
  train_data_dir,
  target_size=(img_height, img_width),
  batch_size=batch_size,
  class_mode='categorical',
  subset='validation')

history = model.fit_generator(train_generator,
                steps_per_epoch = train_generator.samples // batch_size,
                validation_data=validation_generator,
                validation_steps=validation_generator.samples // batch_size,
                epochs=epochs,
                verbose=1,
                callbacks=[csv_logger, checkpointer])

loss, acc = model.evaluate_generator(validation_generator, steps=3, verbose=0)
print('loss: ', loss, 'accuracy: ', acc) # loss:  0.4534463981787364 accuracy:  0.8333333

y_pred = model.predict_generator(validation_generator)
y_pred = np.array([np.argmax(x) for x in y_pred])
y_test = validation_generator.classes

cm = confusion_matrix(y_test, y_pred)
cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
cm.diagonal()
acc_each_class = cm.diagonal()

print('accuracy of each class: \n')
for i in range(len(labels)):
  print(labels[i], ' : ', acc_each_class[i])
print('\n')

'''
accuracy of each class: 

cannoli  :  0.085
dumplings  :  0.065
edamame  :  0.1
falafel  :  0.125
french_fries  :  0.12
grilled_cheese_sandwich  :  0.13
hot_dog  :  0.075
seaweed_salad  :  0.085
tacos  :  0.105
takoyaki  :  0.135
'''

Answer 1

I investigated your issue I think this is due to that you have more than one worker opening the files numerous of times, to fix this issue make sure to set :

shuffle = false

pickle_safe = True

workers = 1

or only evaluate your model via evaluate_generator or predict_generator :

validation_generator2 = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='binary', shuffle=False)

score = model.evaluate_generator(validation_generator, nb_validation_samples/batch_size, workers=12)

scores = model.predict_generator(validation_generator2, nb_validation_samples/batch_size, workers=12)