LSTM timeseries prediction with multiple outputs

Question

I have a dataset with 3 features in a timeseries. The dimension of the dataset is 1000 x 3 (1000 timesteps and 3 features). Basically, 1000 rows and 3 columns

The data looks like this: ABC 131 111 100 131 110 120 131 100 100... 131 100 100 The problem is how to train the first 25 steps and predict the next 25 steps in order to get the output of 3 features predictions which is (A, B and C). I successful train and predict 1-D (1 features(A)) array. But I have no idea how to predict the 3 features using same the dataset.

And I got this error:

Error when checking target: expected dense_1 to have shape (None, 3) but got array with shape (21, 1)

The code as below:

# -*- coding: utf-8 -*-
import numpy as np
import numpy
import matplotlib.pyplot as plt
import pandas
import math

from keras.models import Sequential
from keras.layers import Dense, LSTM, Dropout
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error


# convert an array of values into a dataset matrix

def create_dataset(dataset, look_back=1):
    dataX, dataY = [], []
    for i in range(len(dataset) - look_back - 1):
        a = dataset[i:(i + look_back):]
        dataX.append(a)
        dataY.append(dataset[i + look_back, :])
    return numpy.array(dataX), numpy.array(dataY)



# fix random seed for reproducibility
numpy.random.seed(7)


# load the dataset
dataframe = pandas.read_csv('v77.csv', engine='python',skiprows=0) 
dataset = dataframe.values
print dataset
# normalize the dataset
scaler = MinMaxScaler(feature_range=(0, 1))
dataset = scaler.fit_transform(dataset)

# split into train and test sets
train_size = 10
test_size = 10
train, test = dataset[0:train_size, :], dataset[train_size:train_size+test_size, :]
print (train_size,test_size)

# reshape into X=t and Y=t+1
look_back = 3
trainX, trainY = create_dataset(train, look_back)  
testX, testY = create_dataset(test, look_back)
print trainX

# reshape input to be  [samples, time steps, features]
#trainX = numpy.reshape(trainX, (trainX.shape[0], look_back, 3))
#testX = numpy.reshape(testX, (testX.shape[0],look_back, 3))

# create and fit the LSTM network

model = Sequential()
model.add(LSTM(32, input_shape=(3,3)))
model.add(Dense(3))
model.compile(loss='mean_squared_error', optimizer='adam')
history= model.fit(trainX, trainY,validation_split=0.33, nb_epoch=10, batch_size=16)

# make predictions
trainPredict = model.predict(trainX)
testPredict = model.predict(testX)
# print testPredict
# print np.shape(testPredict)
# Get something which has as many features as dataset
trainPredict_extended = numpy.zeros((len(trainPredict),3))
print trainPredict_extended
print np.shape(trainPredict_extended[:,2])
print np.shape(trainPredict[:,0])
# Put the predictions there
trainPredict_extended[:,2] = trainPredict[:,0]
# Inverse transform it and select the 3rd coumn.
trainPredict = scaler.inverse_transform(trainPredict_extended) [:,2]  
# print(trainPredict)
# Get something which has as many features as dataset
testPredict_extended = numpy.zeros((len(testPredict),3))
# Put the predictions there
testPredict_extended[:,2] = testPredict[:,0]
# Inverse transform it and select the 3rd column.
testPredict = scaler.inverse_transform(testPredict_extended)[:,2]   
# print testPredict_extended

trainY_extended = numpy.zeros((len(trainY),3))
trainY_extended[:,2]=trainY
trainY=scaler.inverse_transform(trainY_extended)[:,2]


testY_extended = numpy.zeros((len(testY),3))
testY_extended[:,2]=testY
testY=scaler.inverse_transform(testY_extended)[:,2]
# print 

# print testY
# calculate root mean squared error
trainScore = math.sqrt(mean_squared_error(trainY, trainPredict))
print('Train Score: %.2f RMSE' % (trainScore))
testScore = math.sqrt(mean_squared_error(testY, testPredict))
print('Test Score: %.2f RMSE' % (testScore))

Sample data: v77.txt

Help Needed. Thanks

Answer 1

Your Y shape does not match up with the last layer in your model. Your Y is in the form of (num_samples, 1) , which means that for every sample it outputs a vector of length 1.

Your last layer, however, is a Dense(3) layer, which outputs (num_samples, 3) , which means that for every sample it outputs a vector of length 3.

Since the output of your neural network and your y-data aren't in the same format, the neural network cannot train.

You can fix this in two ways:

1.Convert the output of your neural network to the shape of your y data by replacing Dense(3) with Dense(1) :

model = Sequential()
model.add(LSTM(32, input_shape=(3,3)))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')history= model.fit(trainX, trainY,validation_split=0.33, nb_epoch=10, batch_size=16)

2.Convert the shape of your y data to the output of your neural network by modifying your create_dataset() function such that all of the features are added to the y instead of just one:

def create_dataset(dataset, look_back=1):
    dataX, dataY = [], []
    for i in range(len(dataset) - look_back - 1):
        a = dataset[i:(i + look_back):]
        dataX.append(a)
        dataY.append(dataset[i + look_back, :])
    return numpy.array(dataX), numpy.array(dataY)

Since you stated that you wanted to predict 3 feature most likely you will be using the second option. Note that the second option does break the last part of your code to extend the y, but your model trains fine.