Keras LSTM Input and Output Dimension Issue

Question

I am trying to create the LSTM model for the multi-steps prediction. Now I am testing the model network setting but it found that it has the dimension issue on the setting.

Here is my testing dataset:

length = 100
df = pd.DataFrame()
df['x1'] = [i/float(length) for i in range(length)]
df['x2'] = [i**2 for i in range(length)]
df['y'] = df['x1'] + df['x2'] 

x_value = df.drop(columns = 'y').values
y_value = df['y'].values.reshape(-1,1)

Here is my t window data building function:

def build_data(x_value, y_value ,n_input, n_output):
    X, Y = list(), list()
    in_start = 0
    data_len = len(x_value)
    # step over the entire history one time step at a time
    for _ in range(data_len):
        # define the end of the input sequence
        in_end = in_start + n_input
        out_end = in_end + n_output
        if out_end <= data_len:
            x_input = x_value[in_start:in_end] # e.g. t0-t3
            X.append(x_input)
            y_output = y_value[in_end:out_end] # e.g. t4-t5
            Y.append(y_output)
        # move along one time step
        in_start += 1
    return np.array(X), np.array(Y)            

X, Y = build_data(x_value, y_value, 1, 2)

The shape of X and Y

X.shape
### (98, 1, 2)
Y.shape
### (98, 2, 1)

For the Model Part,

verbose, epochs, batch_size = 1, 20, 16
n_neurons =  100
n_inputs, n_features  = X.shape[1], X.shape[2]
n_outputs = Y.shape[1]

model = Sequential()
model.add(LSTM(n_neurons, input_shape = (n_inputs, n_features), return_sequences=True))
model.add(TimeDistributed(Dense(1)))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X, Y, epochs=epochs, batch_size=batch_size, verbose=verbose)

It occurred the error: ValueError: Error when checking target: expected time_distributed_41 to have shape (1, 1) but got array with shape (2, 1)

If using the X, Y = build_data(x_value, y_value, 2, 2) ie input window == output window that will be work. But I think it should not contain this constraint.

How can I overcome this issue? ie setting for input window != output window

or any layer or setting I should set?

Answer 1

you encounter a shape mismatch when handling the temporal dimension... the temporal input dim is 1 while you are trying to predict something with a temporal dimension of 2. so you need something in your network that is able to increase from 1 to 2 temporal dimension. I used the Upsampling1D layer, below a full example

# create fake data
X = np.random.uniform(0,1, (98,1,2))
Y = np.random.uniform(0,1, (98,2,1))

verbose, epochs, batch_size = 1, 20, 16
n_neurons =  100
n_inputs, n_features  = X.shape[1], X.shape[2]
n_outputs = Y.shape[1]

model = Sequential()
model.add(LSTM(n_neurons, input_shape = (n_inputs, n_features), return_sequences=True))
model.add(UpSampling1D(n_outputs))
model.add(TimeDistributed(Dense(1)))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X, Y, epochs=epochs, batch_size=batch_size, verbose=verbose)

with input temporal dim > output temporal dim, you can use Lambda or Pooling operation (if the dimension match). below an example with Lambda

X = np.random.uniform(0,1, (98,3,2))
Y = np.random.uniform(0,1, (98,2,1))

verbose, epochs, batch_size = 1, 20, 16
n_neurons =  100
n_inputs, n_features  = X.shape[1], X.shape[2]
n_outputs = Y.shape[1]

model = Sequential()
model.add(LSTM(n_neurons, input_shape = (n_inputs, n_features), return_sequences=True))
model.add(Lambda(lambda x: x[:,-n_outputs:,:]))
model.add(TimeDistributed(Dense(1)))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X, Y, epochs=epochs, batch_size=batch_size, verbose=verbose)