在 VGG16 model 中使用灰度图像时出现“输入不兼容错误”

Question

I am working on an assignment to detect facial emotion of a bunch of gray scale images. I am trying to use VGG16 model for this. I converted the input array of images that are in gray scale to RGB. But when I pass the RGB image array to my model, I am getting incompatibility errors

input_array(gray scale) is of shape 48X48X1

'Converting gray scale to RGB'

'''

input_RGB = np.ndarray(shape=input_array.shape[0],input_array.shape[1],input_array.shape[2], 3),dtype=np.uint8)

input_RGB[:, :, :, 0] = input_array[:, :, :, 0]
input_RGB[:, :, :, 1] = input_array[:, :, :, 0]
input_RGB[:, :, :, 2] = input_array[:, :, :, 0]

''' 'Model definition' '''

from tensorflow.keras.applications.vgg16 import VGG16
base_model = VGG16(weights='imagenet', include_top=False, input_shape = (48, 48, 3))
model2 = Sequential([base_model])
model2.add(Flatten())
model2.add(Dropout(0.25))
model2.add(Dense(64, activation='relu'))
model2.add(Dropout(0.25))
model2.add(Dense(7, activation='softmax'))
model2.compile(optimizer = 'adam', loss='categorical_crossentropy', metrics=['accuracy'])

''' 'Model fitting' '''

history = model.fit(input_RGB, output_array, batch_size = 64, epochs= 20, 
validation_split=0.25,callbacks=[VGG_saved])

'''

'Error message' '''

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
    outputs = self.distribute_strategy.run(
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **
    return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
    return self._call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
    return fn(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:531 train_step  **
    y_pred = self(x, training=True)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:886 __call__
    self.name)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/input_spec.py:216 assert_input_compatibility
    ' but received input with shape ' + str(shape))

ValueError: Input 0 of layer sequential_5 is incompatible with the layer: expected axis -1 of input shape to have value 1 but received input with shape [None, 48, 48, 3]'

Another thing I observed is that when I randomly tried gray scale image arrays as input in model.fit, it did not throw the error however, the validation accuracy is quite low.

Please help

Answer 1

Try change transferring from Sequential to

inputs = keras.Input(shape=(48, 48, 3))
x = base_model(inputs)
x = keras.layers.GlobalAveragePooling2D()(x)
x = keras.layers.Dropout(0.25)(x)
x = keras.layers.Dense(64)(x)
x = keras.layers.Dropout(0.25)(x)
outputs = keras.layers.Dense(7)(x)
model = keras.Model(inputs, outputs)

It might be easier to track what's going wrong.

Also instead of Flatten + AVGPooling you should just use GlobalAveragePooling instead, as it makes using variable size input possible.

More example https://keras.io/guides/transfer_learning/