How to fit the model for multi input problem in keras

Question

I have two sets of data where divided to train and validation subset using train_test_split . I have problem in using model.fit to run the training. can anyone help to put the model.fit in correct order?

(trainY1, valY1, trainX1, valX1) = train_test_split(df, images1, test_size=0.30, random_state=42)
print (np.shape(trainY1),np.shape(valY1),np.shape(trainX1),np.shape(valX1))

(trainY2, valY2, trainX2, valX2) = train_test_split(df, images2, test_size=0.30, random_state=42)
print (np.shape(trainY2),np.shape(valY2),np.shape(trainX2),np.shape(valX2))

result:

(953,) (409,) (953, 16, 16, 4) (409, 16, 16, 4)
(953,) (409,) (953, 16, 16, 4) (409, 16, 16, 4)

model:

v1 = layers.Input(shape = (16,16,4))
cnn1 = layers.Conv2D(filters=32, kernel_size=(3,3), strides=(1,1), padding='same')(v1)
cnn1 = layers.Activation('relu')(cnn1)
cnn1 = layers.MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid')(cnn1)
cnn1 = layers.Flatten()(cnn1)

v2 = layers.Input(shape = (16,16,4))
cnn2 = layers.Conv2D(filters=32, kernel_size=(3,3), strides=(1,1), padding='same')(v2)
cnn2 = layers.Activation('relu')(cnn2)
cnn2 = layers.MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid')(cnn2)
cnn2 = layers.Flatten()(cnn2)

merge  = layers.concatenate([cnn1, cnn2])
dense  = layers.Dense(50, activation='relu')(merge)
output = layers.Dense(1)(dense)

model = Model(inputs=[v1, v2], outputs=output)
model.compile(loss='mse', optimizer='adam')

model.fit([trainX1, trainY1], [trainX2, trainY2],validation_data=([valX1,valY1],[valX2,valY2]), epochs=5, batch_size=32, verbose=1)

error:

input KerasTensor(type_spec=TensorSpec (shape=(None, 16, 16, 4) , dtype=tf.float32 , name='input_24'), name='input_24', description="created by layer 'input_24'") , but it was called on an input with incompatible shape (None, 1).

ValueError: Input 0 of layer conv2d_25 is incompatible with the layer: : expected min_ndim=4, found ndim=2. Full shape received: (None, 1)

Answer 1

You will have to redefine your fit(*) parameters as follows:

model.fit([trainX1, trainX2], [trainY1, trainY2],validation_data=([valX1,valX2],[valY1,valY2]), epochs=5, batch_size=32, verbose=1)

The question is if you also want your model to output two values for trainY1 and trainY2 . Currently, you only have two inputs and one output.