Transfer learning with Inception Resnet v2 (breast cancer) low accuracy

Question

I want to binary classify breast cancer histopathological images from the BreakHis dataset ( https://www.kaggle.com/ambarish/breakhis ) using transfer learning and the Inception Resnet v2. The goal is to freeze all layers and train the fully connected layer by adding two neurons to the model. In particular, initially I want to consider the images related to the magnificant factor 40X (Benign: 625, Malignant: 1370). Here is a summary of what I do:

• I read the images and resize them to 150x150
• I partition the dataset into training, validation and test set
• I load the pre-trained network Inception Resnet v2
• I freeze all the layers I add the two neurons for binary classification (1 = "benign", 0 = "malignant")
• I compile the model using as activation function the Adam method
• I carry out the training
• I make the prediction
• I calculate the accuracy

This is the code:

data = dataset[dataset["Magnificant"]=="40X"]
def preprocessing(dataset, img_size):
    # images
    X = []
    # labels 
    y = []
    
    i = 0
    for image in list(dataset["Path"]):
        # Ridimensiono e leggo le immagini
        X.append(cv2.resize(cv2.imread(image, cv2.IMREAD_COLOR), 
                            (img_size, img_size), interpolation=cv2.INTER_CUBIC))
        basename = os.path.basename(image)
        
        # Get labels
        if dataset.loc[i][2] == "benign":
            y.append(1)
        else:
            y.append(0)
        i = i+1
    return X, y

X, y = preprocessing(data, 150)
X = np.array(X)
y = np.array(y)
# Splitting
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, stratify = y_40, shuffle=True, random_state=1)
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.25, stratify = y_train, shuffle=True, random_state=1) 

conv_base = InceptionResNetV2(weights='imagenet', include_top=False, input_shape=[150, 150, 3])   

# Freezing
for layer in conv_base.layers:
    layer.trainable = False

model = models.Sequential()
model.add(conv_base)
model.add(layers.Flatten())
model.add(layers.Dropout(0.5))
model.add(layers.Dense(1, activation='sigmoid'))

opt = tf.keras.optimizers.Adam(learning_rate=0.0002)

loss = tf.keras.losses.BinaryCrossentropy(from_logits=False)

model.compile(loss=loss, optimizer=opt, metrics = ["accuracy", tf.metrics.AUC()])

batch_size = 32

train_datagen = ImageDataGenerator(rescale=1./255)
val_datagen = ImageDataGenerator(rescale=1./255) 
train_generator = train_datagen.flow(X_train, y_train, batch_size=batch_size) 
val_generator = val_datagen.flow(X_val, y_val, batch_size=batch_size)

callback = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=3)

ntrain =len(X_train)
nval = len(X_val)
len(y_train)
epochs = 70
history = model.fit_generator(train_generator,
                              steps_per_epoch=ntrain // batch_size,
                              epochs=epochs,
                              validation_data=val_generator,
                              validation_steps=nval // batch_size, callbacks=[callback])

This is the output of the training at the last epoch:

Epoch 70/70
32/32 [==============================] - 3s 84ms/step - loss: 0.0499 - accuracy: 0.9903 - auc_5: 0.9996 - val_loss: 0.5661 - val_accuracy: 0.8250 - val_auc_5: 0.8521

I make the prediction:

test_datagen = ImageDataGenerator(rescale=1./255) 
x = X_test
y_pred = model.predict(test_datagen.flow(x))

y_p = []
for i in range(len(y_pred)):
    if y_pred[i] > 0.5:
        y_p.append(1)
    else:
        y_p.append(0)

I calculate the accuracy:

from sklearn.metrics import accuracy_score
accuracy =  accuracy_score(y_test, y_p)
print(accuracy)

This is the accuracy value I get: 0.5459098497495827

Why do I get such low accuracy, I have done several tests but I always get similar results? (HELP ME)

Answer 1

When doing transfer learning, especially with frozen weights, it is extremely important to do the same pre-processing as was used when the network was originally trained.

For the InceptionResNetV2 network the pre-processing type is "tf" in the tensorflow / keras libraries, which corresponds to dividing by 127 then subtracting 1 for the imagenet weights. You are instead dividing by 255.

Fortunately you do not have to wade through the code to find out what function was used, as they are exposed in the API. Simply do

train_datagen = ImageDataGenerator(preprocessing_function=tf.keras.applications.inception_resnet_v2.preprocess_input)

and so on for validation and test