Tensorflow keras 时间序列预测，其中 X 和 y 具有不同的形状

Question

I am trying to do time series prediction with tensorflow and keras with X and y having different dimensions:

X.shape = (5000, 12)
y.shape = (5000, 3, 12)

When I do the following

n_input = 7
generator = TimeseriesGenerator(X, y, length=n_input, batch_size=1)

for i in range(5):
    x_, y_ = generator[i]
    print(x_.shape)
    print(y_.shape)

I get as desired the output

(1, 7, 12)
(1, 3, 12)
(1, 7, 12)
(1, 3, 12)
...

This is because my data is meteorological, I have 5000 days, for training in the array X I use a sliding window of 7 days, with each day containing 12 features (air pressure, temperature, humidity ao). And in the target array y I have sliding windows of 3 days, trying to predict the next 3 days to each window of 7 days.

But then when I try to fit the model I get an error due to the mismatch in the shape of the X and y arrays:

model = Sequential()
model.add(LSTM(4, input_shape=(None, 12)))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
history = model.fit_generator(generator, epochs=3).history

ValueError: A target array with shape (1, 3, 12) was passed for an output of shape (None, 1) while using as loss `mean_squared_error`. This loss expects targets to have the same shape as the output.

So is there a way to adjust the architecture for the mismatch in the dimensions? Or is there a way to reshape X and y to make them work with this architecture? I tried the late reshaping X into (5000, 7, 12) , but this gave also a dimensionality error. Tnx

Answer 1

your generator is correct... it's your network that doesn't work.

you don't handle the dimensionality correctly. you are dealing with sequences so you need to impose return_sequences=True in your LSTM cells. your input has 7 timesteps while your output has 3 timesteps, you have to pass from 7 to 3 (you can do it with pooling and so on).

below a dummy example. I don't use a pooling operation but simply select a part of the sequence in order to get an output of 3 timesteps

X = np.random.uniform(0,1, (5000, 12))
y = np.random.uniform(0,1, (5000, 3, 12))

n_input = 7
generator = tf.keras.preprocessing.sequence.TimeseriesGenerator(X, y, length=n_input, batch_size=32)

model = Sequential()
model.add(LSTM(4, return_sequences=True, input_shape=(n_input, 12)))
model.add(Lambda(lambda x: x[:,-3:,:]))
model.add(Dense(12))
model.compile(loss='mean_squared_error', optimizer='adam')

model.summary()

model.fit(generator, epochs=2)

here an example with pooling operation

model = Sequential()
model.add(LSTM(4, return_sequences=True, input_shape=(n_input, 12)))
model.add(MaxPool1D(2)) # also AvgPool1D is ok
model.add(Dense(12))
model.compile(loss='mean_squared_error', optimizer='adam')

model.summary()
model.fit(generator, epochs=2)

here an example with return_sequences=False and repeat vector

model = Sequential()
model.add(LSTM(4, return_sequences=False, input_shape=(n_input, 12)))
model.add(RepeatVector(3))
model.add(Dense(12))
model.compile(loss='mean_squared_error', optimizer='adam')

model.summary()
model.fit(generator, epochs=2)

Answer 2

Your final (fully connected) layer's shape is (None, 1) and your output's shape is (None, 3, 12). Output shapes of data and the network have to match.

I would use the Functional API and create 3 separate Dense layers and concatenate them. Like this:

inp = tf.keras.Input(shape=(7, 12))
x = tf.keras.layers.LSTM(4)(inp)

y1 = tf.keras.layers.Dense(12)(x)
y2 = tf.keras.layers.Dense(12)(x)
y3 = tf.keras.layers.Dense(12)(x)

y1 = tf.keras.backend.expand_dims(y1, axis=1)
y2 = tf.keras.backend.expand_dims(y2, axis=1)
y3 = tf.keras.backend.expand_dims(y3, axis=1)

output = tf.keras.layers.Concatenate(axis=1)([y1, y2, y3])

mdl = tf.keras.Model(inp, output)
mdl.summary()

Returned: Model Summary