合并两个深度学习模型 VGG16 和 ResNet50 后出错

Question

I have merged two different models namely VGG16 and ResNet50 and given the outputs of the two models as input to another model. I have checked the Layers graph is correct. Before merging the code was running perfectly fine giving correct outputs. I am getting an error:

 "ValueError: Shapes (None, None) and (None, 7, 7, 3) are incompatible" on the line 6 

ValueError                                Traceback (most recent call last)  
<ipython-input-36-620554d0106f> in <module>()  

   4     epochs = 200,  

   5     validation_data = validation_generator,  

----> 6     validation_steps=2  

My code is below:

inputs_2 = keras.Input(shape=(224, 224, 3), name="img")
vgg = VGG16(input_tensor=inputs_2, weights='imagenet', include_top=False) #VGG16(input_shape=IMAGE_SIZE + [3], weights='imagenet', include_top=False)
for layer in vgg.layers:
    layer.trainable = False
#x = Flatten()(vgg.output)

resnet = ResNet50(input_tensor=inputs_2, weights='imagenet', include_top=False) #ResNet50(input_shape=IMAGE_SIZE + [3], weights='imagenet', include_top=False)
for layer in resnet.layers:
    layer.trainable = False
#y = Flatten()(resnet.output)

#concatenated_output= layers.add([x, y])
mergedOutput = Concatenate()([vgg.output, resnet.output])
x = layers.Dense(256, activation="relu")(mergedOutput)
prediction = Dense(3, activation='softmax')(x)
model = Model(inputs=vgg.input, outputs=prediction)


model.compile(loss="categorical_crossentropy",optimizer='adam',metrics=['accuracy'])

keras.utils.plot_model(model, "mini_resnet.png", show_shapes=True)  
    train_datagen = image.ImageDataGenerator(
    rescale = 1./255,
    shear_range = 0.2,
    zoom_range = 0.2,
    horizontal_flip = True,
)

test_dataset = image.ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
    TRAIN_PATH,
    color_mode = "rgb",
    target_size = (224,224),
    batch_size = 32,
    class_mode = 'categorical'
    )
print(train_generator.class_indices)

validation_generator = test_dataset.flow_from_directory(
    VAL_PATH,
    color_mode = "rgb",
    target_size = (224,224),
    batch_size = 32,
    class_mode = 'categorical')

history = model.fit_generator(
    train_generator,
    steps_per_epoch=8,
    epochs = 200,
    validation_data = validation_generator,
    validation_steps=2
)

Graph of merged models

I am using one-hot encoding and have correctly written the parameters.

Answer 1

Try this:

model = Model(inputs=inputs_2, outputs=prediction)

If you set your inputs with vgg.input , you don't feed the Resnet part so it outputs nothing.