在具有Tensorflow后端的Keras上，在不同的输入部分上并行拟合LSTM和一些密集层

Question

I am working on a regression forecast where I have some complex 3D sequences and some features explaining some key characteristics of the sequences. They are held on two matrices of such shapes:

X1.shape, X2.shape
((9000, 300, 3), (9000, 106))

I want to feed them to a Model instance where the X1 matrix is dealt by a LSTM and the X2 matrix by a couple of dense layers. My plan is to merge them before the output layer.

I was planning to train by:

model.fit(zip(X1, X2), y, batch_size=BATCH, epochs=EPOCHS, validation_split=0.2)

How to build the model in order to receive the two matrices and deal them separately?

At the moment I just have my standard model for LSTM only:

def model(sample_size=300, axis=3):
    inp=Input(shape=(sample_size, axis))
    x=LSTM(50, return_sequences=True)(inp)
    x=GlobalMaxPool1D(x)
    x=Dense(1)(x)
    model=Model(inputs=inp, ouputs=x)
    model.compile(loss='mean_squared_error', optimizer='adam',
                  metrics= ['mae'])
   return model

Answer 1

I think this should work

# First input
input1=Input(shape=(300,3))
x=LSTM(50, return_sequences=True)(input1)
x=GlobalMaxPool1D(x)
x=Dense(n)(x)

# Second Input
input2=Input(shape=(106))
y=Dense(n)(input2)

# Merge
merged=Concatenate([x,y])
merged=Dense(1)(merged)

# Define model with two inputs
model=Model(inputs=[input1,input2],outputs=merged)

Both the models should have same output space before merging. Then, you can pass a list of inputs and Keras will pass it at the appropriate places

model.fit([X1,X2],....)