Keras functional API multiple input LSTM

Question

I am trying to build two inputs LSTM model using Keras functional API.

Here is the code I used:

inp_1 = 100
inp_2 = 100
out_1 = 10
N_FEATURES = 1


Input_1 = Input(shape=(inp_1, N_FEATURES), name='Input_1')
LSTM_1 = LSTM(name='LSTM_1', units=128)(Input_1)
Dense_1 = Dense(name='Dense_1', units=128)(LSTM_1)
Input_2 = Input(shape=(inp_2,), name='Input_2')
Dense_2 = Dense(name='Dense_2', units=128)(Input_2)
merge_2 = concatenate([Dense_1, Dense_2])

RepeatVector_1 = RepeatVector(out_1, name='RepeatVector_1')(merge_2)
LSTM_2 = LSTM(name='LSTM_2', units=128)(RepeatVector_1)
output = TimeDistributed(Dense(1,activation='linear'))(LSTM_2)

model = Model(inputs=[Input_1, Input_2], output=    output)

model.compile()

However, I got the following error that I do not understand:

assert len(input_shape) >= 3 AssertionError

for line:

output = TimeDistributed(Dense(1,activation='linear'))(LSTM_2)

The layer already accepts a tensor of length 128. What am I missing here?

Answer 1

As the document says

The input should be at least 3D , and the dimension of index one will be considered to be the temporal dimension.

TimeDistributed layer applies a layer to every temporal slice of an input. So you should set return_sequences=True to return the complete time series output in upper layer LSTM_2 .

LSTM_2 = LSTM(name='LSTM_2', units=128,return_sequences=True)(RepeatVector_1)
# LSTM_2 output shape =(?, 10, 128)