这是过拟合的情况吗？ 我该怎么做才能修复我的 model？

Question

I am quite new to CNN, I have been working on a CNN model that trains on a small dataset of 506 images. (306 for category 0 and 200 for Category 1). I did all the work on Google Colab. I picked, mixed and played around with the model, borrowing from many examples I can find online. I somehow managed to get the model's training accuracy to reach 0.8022 and val_acc 0.8431 The model.evaluate gives: loss: 0.6061 - accuracy: 0.8755

Now, my problem is, I tried doing model.predict with my model. The prediction is horrible, every image of each category 0 and 1, my model classified it as category 0. All the prediction result is just 0. (I fed the model equal amounts of category 0 and 1)

Here is my code:

import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
import cv2
import os
import glob
from tensorflow import keras
from keras import layers
from keras.models import Sequential, model_from_json
from keras.layers import Dense, Dropout, Flatten, BatchNormalization, Activation
from keras.layers import Conv2D, MaxPooling2D
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D
from keras.preprocessing import image
from keras.regularizers import l2 


DATADIR = r"/content/gdrive/MyDrive/Dataset/Training"
CATEGORIES = ['Abnormal', 'Normal']


training_data = []

def create_training_data():
    for category in CATEGORIES:
        path = os.path.join(DATADIR, category)
        class_num = CATEGORIES.index(category)
        for img in os.listdir(path):
            try:
                img_array = cv2.imread(os.path.join(path,img), cv2.IMREAD_GRAYSCALE)
                new_array = cv2.resize(img_array, (1000, 400))
                training_data.append([new_array, class_num])
            except Exception as e:
                pass
                
create_training_data()

print(len(training_data))

import random
random.shuffle(training_data)

for sample in training_data:
    print(sample[1])

X = []
y = []

for features, label in training_data:
    X.append(features)
    y.append(label)
X = np.array(X).reshape(-1, 1000, 400, 1)
y = np.array(y)

import pickle

pickle_out = open("X.pickle","wb")
pickle.dump(X, pickle_out)
pickle_out.close()

pickle_out = open("y.pickle","wb")
pickle.dump(y, pickle_out)
pickle_out.close()
pickle_in = open("X.pickle", "rb")
X = pickle.load(pickle_in)

X = pickle.load(open("X.pickle", "rb"))
y = pickle.load(open("y.pickle", "rb"))

X = X/255.0



model = Sequential()


model.add(Conv2D(128, (3,3), activation = 'relu', input_shape=(1000,400,1), kernel_regularizer=l2(0.001)))
model.add(Dropout(0.5))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3),activation = 'relu',kernel_regularizer=l2(0.001)))
model.add(Dropout(0.5))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3),activation = 'relu',kernel_regularizer=l2(0.001)))
model.add(Dropout(0.5))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(128 , activation='relu',kernel_regularizer=l2(0.001)))
model.add(Dropout(0.4))
model.add(Dense(1, activation='sigmoid',kernel_regularizer=l2(0.001)))
model.summary()

model.compile(loss='binary_crossentropy',

              optimizer=keras.optimizers.Adam(learning_rate=0.0001),

              metrics=['accuracy'])

callback = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=2)

model.fit(X, y, shuffle=True, batch_size = 7, epochs = 20, validation_split = 0.1, callbacks=[callback])
model.evaluate(X, y)

Here is the model summary and training:

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 998, 398, 128)     1280      
                                                                 
 dropout (Dropout)           (None, 998, 398, 128)     0         
                                                                 
 max_pooling2d (MaxPooling2D  (None, 499, 199, 128)    0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 497, 197, 64)      73792     
                                                                 
 dropout_1 (Dropout)         (None, 497, 197, 64)      0         
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 248, 98, 64)      0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 246, 96, 32)       18464     
                                                                 
 dropout_2 (Dropout)         (None, 246, 96, 32)       0         
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 123, 48, 32)      0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 188928)            0         
                                                                 
 dense (Dense)               (None, 128)               24182912  
                                                                 
 dropout_3 (Dropout)         (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 1)                 129       
                                                                 
=================================================================
Total params: 24,276,577
Trainable params: 24,276,577
Non-trainable params: 0
_________________________________________________________________
Epoch 1/20
65/65 [==============================] - 35s 338ms/step - loss: 0.9851 - accuracy: 0.5692 - val_loss: 0.9020 - val_accuracy: 0.6275
Epoch 2/20
65/65 [==============================] - 21s 328ms/step - loss: 0.8785 - accuracy: 0.5956 - val_loss: 0.8668 - val_accuracy: 0.6275
Epoch 3/20
65/65 [==============================] - 21s 331ms/step - loss: 0.8428 - accuracy: 0.5956 - val_loss: 0.8425 - val_accuracy: 0.6275
Epoch 4/20
65/65 [==============================] - 22s 333ms/step - loss: 0.8229 - accuracy: 0.5934 - val_loss: 0.8219 - val_accuracy: 0.6275
Epoch 5/20
65/65 [==============================] - 22s 335ms/step - loss: 0.7974 - accuracy: 0.6154 - val_loss: 0.8033 - val_accuracy: 0.6275
Epoch 6/20
65/65 [==============================] - 22s 334ms/step - loss: 0.7646 - accuracy: 0.5956 - val_loss: 0.7968 - val_accuracy: 0.6275
Epoch 7/20
65/65 [==============================] - 22s 334ms/step - loss: 0.7629 - accuracy: 0.6264 - val_loss: 0.7898 - val_accuracy: 0.6275
Epoch 8/20
65/65 [==============================] - 22s 334ms/step - loss: 0.7467 - accuracy: 0.6176 - val_loss: 0.7761 - val_accuracy: 0.6667
Epoch 9/20
65/65 [==============================] - 22s 335ms/step - loss: 0.7191 - accuracy: 0.6286 - val_loss: 0.7667 - val_accuracy: 0.6667
Epoch 10/20
65/65 [==============================] - 22s 334ms/step - loss: 0.7222 - accuracy: 0.6527 - val_loss: 0.7674 - val_accuracy: 0.6863
Epoch 11/20
65/65 [==============================] - 22s 335ms/step - loss: 0.6664 - accuracy: 0.6769 - val_loss: 0.7440 - val_accuracy: 0.6863
Epoch 12/20
65/65 [==============================] - 22s 334ms/step - loss: 0.6688 - accuracy: 0.6769 - val_loss: 0.7550 - val_accuracy: 0.6471
Epoch 13/20
65/65 [==============================] - 22s 334ms/step - loss: 0.6535 - accuracy: 0.7077 - val_loss: 0.7611 - val_accuracy: 0.7059
Epoch 14/20
65/65 [==============================] - 22s 334ms/step - loss: 0.6221 - accuracy: 0.7187 - val_loss: 0.7487 - val_accuracy: 0.7255
Epoch 15/20
65/65 [==============================] - 22s 334ms/step - loss: 0.5909 - accuracy: 0.7473 - val_loss: 0.7286 - val_accuracy: 0.7451
Epoch 16/20
65/65 [==============================] - 22s 335ms/step - loss: 0.5814 - accuracy: 0.7516 - val_loss: 0.7235 - val_accuracy: 0.7647
Epoch 17/20
65/65 [==============================] - 22s 334ms/step - loss: 0.5653 - accuracy: 0.7714 - val_loss: 0.7264 - val_accuracy: 0.8039
Epoch 18/20
65/65 [==============================] - 22s 334ms/step - loss: 0.5367 - accuracy: 0.8000 - val_loss: 0.6934 - val_accuracy: 0.8039
Epoch 19/20
65/65 [==============================] - 22s 334ms/step - loss: 0.5147 - accuracy: 0.8022 - val_loss: 0.7242 - val_accuracy: 0.7255
Epoch 20/20
65/65 [==============================] - 22s 335ms/step - loss: 0.4832 - accuracy: 0.7824 - val_loss: 0.6765 - val_accuracy: 0.8431
16/16 [==============================] - 15s 747ms/step - loss: 0.6061 - accuracy: 0.8755
[0.6060804724693298, 0.8754940629005432]

Also, this is how i use the model.predict:

for i in os.listdir(testdata):
    img = image.load_img(testtest+'//'+ i, color_mode = "grayscale", target_size=(1000,400))
    xtest = image.img_to_array(img)
    xtest = np.expand_dims(xtest, axis = 0)
    images = np.vstack([xtest])
    val = model.predict(images)
    val2 = model.evaluate(images, verbose = 0)
    print (np.argmax(val))

Answer 1

The prediction is horrible, every image of each category 0 and 1, my model classified it as category 0.

This is actually very common in binary classification. What is the distribution in your training set? Given that your training accuracy is 0.8022, it sounds like 80.22% of your training data is class 0, and the model simply learned to pick the majority class.