Im getting ValueError when trying to load my own weights for a transfer learning task

Question

Hi I am trying to do Transfer Learning in Keras and I am trying to load weights into a new model that I have self trained from a different task.

I have trained my own set of weights from another task. This other task, however, is a binary classification problem while my new problem is a multi-label classification problem.

I got my first set of weights doing this:

n_classes = 1
epochs = 100
batch_size = 32
input_shape = (224, 224, 3)

base_model = MobileNetV2(input_shape=input_shape, weights= None, include_top=False)
x = GlobalAveragePooling2D()(base_model.output)

output = Dense(n_classes, activation='sigmoid')(x)
model = tf.keras.models.Model(inputs=[base_model.input], outputs=[output])

opt = optimizers.Adam(lr = 0.001)

model.compile(optimizer=opt, loss='binary_crossentropy', metrics=['accuracy'])

...
...

history = model.fit(train_generator, epochs=epochs, 
                    steps_per_epoch=step_size_train,verbose=1,
                    validation_data=valid_generator,
                    validation_steps=STEP_SIZE_VALID,
                    class_weight=class_weights,
                    )


model.save_weights("initial-weights.h5")

But when I try to load these weights into my new model:

weights_path = 'initial-weights.h5'

n_classes = 14
epochs = 1000
batch_size = 32
input_shape = (224, 224, 3)

base_model = MobileNetV2(input_shape=input_shape, weights= None, include_top=False)
x = GlobalAveragePooling2D()(base_model.output)


output = Dense(n_classes, activation='sigmoid')(x)
model = tf.keras.models.Model(inputs=[base_model.input], outputs=[output])

opt = optimizers.Adam(lr = 0.001)

model.compile(optimizer=opt, loss='binary_crossentropy', metrics=['accuracy'])

model.load_weights(weights_path)

I get the following error:

ValueError: Shapes (1280, 14) and (1280, 1) are incompatible

I understand that based on the error, it is very likely to be due to the difference in the number of classes, but from what I know about transfer learning, it is possible to transfer weights from different tasks even if the number of classes are different (like how ImageNet weights are used for tasks that have different number of classes).

How do I initialize my own set of custom weights that are trained from a different task that has a different number of classes?

Answer 1

I think that the best approach is to transfer the weights for all layers except the last (ie. the feature extraction part). Then you can freeze all the transferred weights, and train the the model again, where only the weights on the last layer (ie. classification layer) will be trained.