Keras LSTM TensorFlow Error: 'Shapes must be equal rank, but are 1 and 0'

Question

I'm trying to create a Keras LSTM (Please note that I am new to LSTMs and RNNs in Keras). The neural network is supposed to take an input of 4116 values, and output 4116 values. This is to be done for 288 timesteps. I have 27 such timesteps (I realize this will likely lead to overfitting; I have a larger dataset, but first want to test my code with just 27 training examples).

The training data is stored in two numpy arrays x and y . These variables have a shape of (27, 288, 4116) .

My code:

datapoints = data.get.monthPoints(2, year)
x, y = datapoints[:-1], datapoints[1:]
del datapoints

input_shape = x.shape[1:]
output_shape = y.shape[1:]

checkpoint = ModelCheckpoint('model/files/alpha.h5', monitor='val_loss', verbose=1, save_best_only=True, mode='auto', period=1)
early = EarlyStopping(monitor='val_loss', min_delta=0, patience=1, verbose=1, mode='auto')

model = Sequential()
model.add(LSTM(5488, activation='relu', input_shape=input_shape))
model.add(RepeatVector(output_shape))
model.add(LSTM(5488, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(output_shape)))

model.compile(loss='mse', optimizer='adam')
model.fit(x, y, epochs=100, batch_size=8, callbacks = [checkpoint, early])

When I run the program, I get the following error(s):

tensorflow.python.framework.errors_impl.InvalidArgumentError: Shapes must be equal rank, but are 1 and 0
        From merging shape 1 with other shapes. for 'repeat_vector/stack_1' (op: 'Pack') with input shapes: [], [2], []

and

During handling of the above exception, another exception occurred:
ValueError: Shapes must be equal rank, but are 1 and 0
        From merging shape 1 with other shapes. for 'repeat_vector/stack_1' (op: 'Pack') with input shapes: [], [2], []

I've seen a few other similar questions like this and this , but they haven't offered solutions that fix my problem or the solutions have been unclear.

I guess my problem has something to do with me structuring the network incorrectly or formatting my data incorrectly.

Any insight would me much appreciated.

Thank you.

Answer 1

You probably want to repeat the output of first LSTM layer as much as the number of timesteps in model's output sequence (ie y ). Therefore, it should be:

model.add(RepeatVector(y.shape[1]))

Answer 2

There are two problems in your code. First, in RepeatVector you are sending a list by passing y.shape[1:]. In RepeatVector you should be sending an integer.

Second problem is in TimeDistributed . Send in it the number of times you would like your second dimension to be repeated at. So your code should be:

repeat_vector_units = x.shape[1]
output_units = x.shape[2]

model = Sequential()
model.add(LSTM(5488, activation='relu', input_shape=input_shape))
model.add(RepeatVector(repeat_vector_units))        ### Change here
model.add(LSTM(5488, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(output_units)))     #### Change here