用于图像分类的Keras CNN模型没有很好地概括

Question

I want to implement a model in keras for sentiment classification(anger or non anger) based on spectograms. I have generated the spectograms using the audio dataset from Friends. Each spectogram has a length of 8 seconds. In total, I have 9117 train samples, 1006 validation samples and 2402 test samples.

I use a relatively simple CNN architecture and I tried different combinations of it + optimizer + learning rate + batch size but none of the results seem to generalize well...The loss decreases nicely till a certain point but the validation loss increases by each epoch.

This is the model I am using:

model = Sequential()
        model.add(Convolution2D(filters=32, kernel_size=3, strides=1,input_shape=input_shape, activation='relu', padding="same"))
        model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))  

        # model.add(ZeroPadding2D((1, 1)))
        model.add(Convolution2D(filters=64, kernel_size=3, strides=1, activation='relu', padding="same"))
        model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2))) 

        model.add(Convolution2D(filters=128, kernel_size=3, strides=1, activation='relu', padding="same"))
        model.add(MaxPooling2D((2, 2), strides=(2, 2))) 

        model.add(Flatten())
        model.add(Dense(128, activation='relu'))
        model.add(Dropout(0.5))
        model.add(Dense(128, activation='relu'))
        model.add(Dropout(0.5))
        model.add(Dense(classes, activation='sigmoid')) #output layer

This is how I load the images:

img_rows = 120
img_cols = 160

train_datagen = ImageDataGenerator(rescale=1./255)
validation_datagen = ImageDataGenerator(rescale=1./255)
test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
            SPECTOGRAMS_DIRECTORY + TRAIN_SUBDIR,
            target_size=(img_cols, img_rows),
            batch_size=batch_size,
            class_mode='binary')

validation_generator = validation_datagen.flow_from_directory(
                    SPECTOGRAMS_DIRECTORY + VALIDATION_SUBDIR,
                    target_size=(img_cols, img_rows),
                    batch_size=batch_size,
                    class_mode='binary')

test_generator = test_datagen.flow_from_directory(
                SPECTOGRAMS_DIRECTORY + TEST_SUBDIR,
                target_size=(img_cols, img_rows),
                batch_size=1,  
                class_mode='binary',
                shuffle=False)

input_shape=(img_cols, img_rows, channels)
opt = SGD(lr=0.001)
model.compile(loss='binary_crossentropy',
              optimizer=opt,
              metrics=['accuracy'])

history = model.fit_generator(
        train_generator,
        steps_per_epoch=nb_train_samples // batch_size,
        epochs=epochs,
        validation_data=validation_generator,
        validation_steps=nb_validation_samples // batch_size,
        verbose=2)

##EVALUATE
print("EVALUATE THE MODEL...")
score = model.evaluate_generator(generator=validation_generator,
                         steps=nb_validation_samples // batch_size)

The spectograms look like this:

As I said, I tried using different combinations of batch size (16,32,64), SGD with 0.001 learning rate, Adam with 0.0001 learning rate, but for each combination the training loss goes down while the validation loss goes up.

Answer 1

Model seems to be over-fitting. You can try the below approaches to overcome this issue.

1. If possible try to gather more data or you can use data augmentation techniques to increase the number of samples.

2. You can use dropout in Keras to reduce the over-fitting. (Looks like you have already added dropout, you can try tuning the values)

Thank you