LSTM input_shape in keras

Question

I am implementing a LSTM network in keras Spyder according to a Udemy tutorial. In the tutorial video the code is running but when I run it on my own, which step by step is precisely implemented the same as the tutorial video code, it gets an error which is:

Error when checking target: expected dense_26 to have 3 dimensions, but got array with shape (1198, 1)

Does it relate to the version of Spyder? The size of x_train is (1198,60,1) and y_train size is (1198,)

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

#Importing the training set
dataset_train = pd.read_csv('E:\\downloads_1\\Recurrent_Neural_Networks\\Google_Stock_Price_Train.csv')
training_set = dataset_train.iloc[:,1:2].values
#Feature Scaling 
from sklearn.preprocessing import MinMaxScaler
sc = MinMaxScaler(feature_range=(0,1))
training_set_scaled = sc.fit_transform(training_set)
#Creating a Data Structure with 60 timesteps and 1 output
x_train = []
y_train = []
for i in range(60,1258):
    x_train.append(training_set_scaled[i-60:i , 0])
    y_train.append(training_set_scaled[i , 0])
x_train , y_train = np.array(x_train) , np.array(y_train)

#Reshaping
x_train = np.reshape(x_train , (x_train.shape[0],x_train.shape[1], 1))



#Part 2_Building the Rnn

#importing te keras Libraries and Packages
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
from keras.layers import LSTM


#initializing the RNN
regressor = Sequential()

#Adding the first layer LSTM and some Dropout Regularisations
regressor.add(LSTM(units = 50, return_sequences= True, input_shape = (x_train.shape[1], 1)))
regressor.add(Dropout(0.2))

#Adding the second layer LSTM and some Dropout Regularisations
regressor.add(LSTM(units = 50, return_sequences= True))
regressor.add(Dropout(0.2))

#Adding the third layer LSTM and some Dropout Regularisations
regressor.add(LSTM(units = 50, return_sequences= True))
regressor.add(Dropout(0.2))

#Adding the fourth layer LSTM and some Dropout Regularisations
regressor.add(LSTM(units = 50, return_sequences= True))
regressor.add(Dropout(0.2))

#Adding the output layer
regressor.add(Dense(units = 1))

#compiling the RNN
regressor.compile(optimizer= 'adam',  loss = 'mean_squared_error')

#Fitting the RNN TO the training set`enter code here`
regressor.fit(x_train, y_train, epochs = '100', batch_size = 32 )

Answer 1

Your LSTM is returning a sequence (ie return_sequences=True ). Therefore, your last LSTM layer returns a (batch_size, timesteps, 50) sized 3-D tensor. Then the dense layer returns a 3-D predictions (ie (batch_size, time steps, 1) ) array.

But it appears you are feeding in a 2-D input as the outputs (ie 1192x1 ). So this needs to be a (1192, 60, 1) array for your model to use the labels.

This really depends on your problem. The other alternative is, if you really have only 1 output value per data point, you need to use return_sequences=False on the last LSTM.

#initializing the RNN
regressor = Sequential()

#Adding the first layer LSTM and some Dropout Regularisations
regressor.add(LSTM(units = 50, return_sequences= True, input_shape = (x_train.shape[1], 1)))
regressor.add(Dropout(0.2))

#Adding the second layer LSTM and some Dropout Regularisations
regressor.add(LSTM(units = 50, return_sequences= True))
regressor.add(Dropout(0.2))

#Adding the third layer LSTM and some Dropout Regularisations
regressor.add(LSTM(units = 50, return_sequences= True))
regressor.add(Dropout(0.2))

#Adding the fourth layer LSTM and some Dropout Regularisations
regressor.add(LSTM(units = 50, return_sequences= False))
regressor.add(Dropout(0.2))

PS: Also something you should be aware is that your Dense layer is actually 3-D. This is fine if you were going for this. But the typical way of using a Dense layer in a time-series model is to use a TimeDistributed layer like follows.

regressor.add(TimeDistirbuted(Dense(units = 1)))