VGG16 对痤疮分级的准确度极低

Question

I am trying to build an ance grading classifier with 3 classes (Level 0, Level 1, Level 2). I do not have a lot of images in the dataset (around 500 in each class). Therefore I used the VGG16 pre-trained model. However, the accuracy is really low (~0.33) and barely increased along with the training.

# Load VGG16 model
vgg_model = VGG16(weights="imagenet", 
                  include_top=False, 
                  input_tensor=Input(shape=(224,224,3)))
vgg_model.summary()

#make the model layers untrainable
for layer in vgg_model.layers:
    layer.trainable = False

# Add output layer
output_model = vgg_model.output
output_model = layers.Dropout(0.25)(output_model)
output_model = layers.Flatten()(output_model)
output_model = layers.Dense(128,activation="relu")(output_model)
output_model = layers.Dropout(0.5)(output_model)
output_model = layers.Dense(3,activation="softmax")(output_model)

vggmodel = models.Model(inputs=vgg_model.input, outputs=output_model)

vggmodel.summary()


# Image augmentation on training set
train_datagen = ImageDataGenerator(
    rotation_range = 40,
    width_shift_range = 0.2,
    height_shift_range = 0.2,
    shear_range = 0.15,
    rescale = 1./255,
)

# only rescale on validation set
validate_datagen = ImageDataGenerator(
    rescale = 1./255
)

#set size of batches of data to 64
batch_size = 64
#resizing to 224x224
target_size = (224, 224)

# initialize the training data augmentation object
train_generator = train_datagen.flow_from_directory(directory=training_path, class_mode="categorical",
                                                  batch_size=batch_size, target_size=target_size, color_mode='rgb', shuffle= True)
validation_generator = validate_datagen.flow_from_directory(directory=validation_path, class_mode="categorical",
                                                  batch_size=batch_size, target_size=target_size, color_mode='rgb', shuffle= True)

vggmodel.compile(loss="categorical_crossentropy", optimizer=SGD(0.01),metrics=["accuracy"])

earlystopping = keras.callbacks.EarlyStopping(monitor ="val_loss",  
                                        mode ="min", patience = 2,  
                                        restore_best_weights = True) 

vggmodel.fit(train_generator, steps_per_epoch=int(1166/batch_size), epochs= 100,
                        validation_data=validation_generator, validation_steps=5, callbacks=[earlystopping])

I have also tried building my own model but the performance is similar.

input_shape = (224,224,3)
cnn_model = models.Sequential()
cnn_model.add(layers.Conv2D(64, (3, 3), activation='relu',
                               input_shape=input_shape))
cnn_model.add(layers.Conv2D(64, (3, 3), activation='relu',
                                input_shape=input_shape))
cnn_model.add(layers.MaxPool2D((2, 2)))
cnn_model.add(layers.Conv2D(64, (3, 3), activation='relu',
                                input_shape=input_shape))
cnn_model.add(layers.MaxPool2D((2, 2)))
cnn_model.add(layers.Dropout(0.25))
cnn_model.add(layers.Flatten())
cnn_model.add(layers.Dense(128, activation='relu'))
cnn_model.add(layers.Dropout(0.5))
cnn_model.add(layers.Dense(3, activation='softmax'))
cnn_model.summary()

cnn_model.compile(loss="categorical_crossentropy", optimizer=SGD(0.01),metrics=["accuracy"])


earlystopping = keras.callbacks.EarlyStopping(monitor ="val_loss",  
                                        mode ="min", patience = 2,  
                                        restore_best_weights = True) 

cnn_model.fit(train_generator, steps_per_epoch=int(1248/batch_size), epochs= 15,
                        validation_data=validation_generator, validation_steps=2)

What did I do wrong? Sorry, I am a beginner:/

Answer 1

1. Use the max-pooling layer, instead of a flatten layer.
2. This user had to reduce learning rate to 0.001, to get acceptable results.
3. Use smaller dropout rate. .2, .3 for example.
4. 33% is just random guessing. Print some images and see that their labels are correct.
5. Check the output of your model on some sample image. Is it too close to 1 or 0 in some columns? Does it seem just random?

Do these and you might catch the bug.