Time Series Prediction with LSTM in Keras

Question

I am a beginner in Deep learning. I am implementing LSTM using the keras library, to predict the weather data, I have train and test data. After removing some variables, my imput data has following shape.

('X_train', (117, 22))
('y_train', (117,))
('X_test', (13, 22))
('y_test', (13,))

Now, I am feeding this data into the LSTM code below, but could not figure out the input shape, where I am facing the trouble. Given below is the full code for the LSTM that I am applying.

import os
print os.getcwd()
import pandas
import numpy
import matplotlib.pyplot as plt
import math
from sklearn.metrics import mean_squared_error


train = pandas.read_excel('./data/train.xlsx', sheetname = 'temp4')
print train.head()
print train.shape

test = pandas.read_excel('./data/test.xlsx', sheetname = 'temp4')
print test.head()
print test.shape

# lagsp has 7 misssing values in train data and rest is tha in all entries and also drop un-necessary variable
train = train.drop(['WEEK_NBR', 'DOS_YEAR', 'sorted row','lagsp'], axis = 1)
test = test.drop(['WEEK_NBR', 'DOS_YEAR', 'sorted row','lagsp'], axis = 1)

print train.shape
print test.shape

train = train.values
test = test.values

X_train = train[:,0:22]
y_train = train[:,22]

X_test = test[:,0:22]
y_test = test[:,22]

print("X_train", X_train.shape)
print("y_train", y_train.shape)
print("X_test", X_test.shape)
print("y_test", y_test.shape)




timesteps = X_train.shape[0]
features = X_train.shape[1]

X_train = X_train.reshape(1, timesteps, features)
X_test = X_test.reshape(1, X_test.shape[0], X_test.shape[1])



print 'timesteps', timesteps
print 'features', features



numpy.random.seed(7)


from keras.models import Sequential
from keras.layers import Dense, Dropout


###########################
#   RNN
###########################
from keras.layers.recurrent import LSTM

d = 0.2
rnn_model = Sequential()
rnn_model.add(LSTM(64, input_shape= (117,22), return_sequences=True))
rnn_model.add(Dropout(d))
rnn_model.add(Dense(16,kernel_initializer='uniform',activation='relu'))
rnn_model.add(Dense(1,kernel_initializer='uniform',activation='linear'))
rnn_model.compile(loss='mse',optimizer='rmsprop',metrics=['accuracy'])

#batch_input_shape=(batch_size, timesteps, data_dim)

rnn_model.fit(
    X_train,
    numpy.array(y_train),
    batch_size=10,
    epochs=10)

# make predictions
trainPredict_rnn = rnn_model.predict(X_train)
testPredict_rnn = rnn_model.predict(X_test)
# calculate root mean squared error
trainScore_rnn = math.sqrt(mean_squared_error(y_train, trainPredict_rnn))
print('Train Score: %.2f RMSE' % (trainScore_rnn))
testScore_rnn = math.sqrt(mean_squared_error(y_test, testPredict_rnn))
print('Test Score: %.2f RMSE' % (testScore_rnn))

# plot predictions

plt.figure(figsize=(20,10))
plt.plot(y_train) # blue # orange
plt.plot(trainPredict_rnn)
plt.show()

plt.plot(y_test) # blue # orange
plt.plot(testPredict_rnn)
plt.show()

This is the error, i am geting after running the code where, the model is fitting X_train and y_train, with above input shape.

Traceback (most recent call last):
  File "/home/shivampanchal/PycharmProjects/WeatherPrediction/try.py", line 81, in <module>
    epochs=10)
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 856, in fit
    initial_epoch=initial_epoch)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1429, in fit
    batch_size=batch_size)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1309, in _standardize_user_data
    exception_prefix='target')
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 127, in _standardize_input_data
    str(array.shape))
ValueError: Error when checking target: expected dense_2 to have 3 dimensions, but got array with shape (117, 1)

Answer 1

X_train needs to be three-dimensional. When specifying the input_shape in your first layer, you are specificying (timesteps, features) . Then, when passing actual data to fit() , you need to pass in a three-dimensional array where each sample has the shape (timesteps, features) .

timesteps = X_train.shape[0]
features = X_train.shape[1]

X_train = X_train.reshape(1, timesteps, features)

This leaves you with a single training sample, though, which I doubt is what you want. Without knowing what your data actually looks like it is hard to be of further help! It seems more likely you'd want to break down your dataset into sequences of some fixed amount of timesteps. Additionally, you'll want to apply similar transformations to X_test .