LSTM神经网络输入/输出尺寸错误

Question

I am fairly new to TensorFlow and LSTM architecture. I have an issue with figuring out input and output (x_train,x_test,y_train,y_test) for my dataset.

Shape of my inputs originally:

• X_train: (366,4)
• X_test: (104,4)
• Y_train: (366,)
• Y_test: (104,)

The Ytrain and Ytest are a series of stock prices. The Xtrain and Xtest are four features that I want to learn to predict the stock prices.

# Splitting the training and testing data

train_start_date = '2010-01-08'
train_end_date = '2017-01-06'
test_start_date = '2017-01-13'
test_end_date = '2019-01-04'

train = df.ix[train_start_date : train_end_date]
test = df.ix[test_start_date:test_end_date]


X_test = sentimentScorer(test)
X_train = sentimentScorer(train)

Y_test = test['prices'] 
Y_train = train['prices']

#Conversion in 3D array for LSTM INPUT

X_test = X_test.reshape(1, 104, 4)
X_train = X_train.reshape(1, 366, 4)





model = Sequential()

model.add(LSTM(128, input_shape=(366,4), activation='relu', 
return_sequences=True))
model.add(Dropout(0.2))

model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.1))

model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))

model.add(Dense(10, activation='softmax'))

opt = tf.keras.optimizers.Adam(lr=0.001, decay=1e-6)

# Compile model
model.compile(
    loss='sparse_categorical_crossentropy',
    optimizer=opt,
    metrics=['accuracy'],
)

model.fit(X_train,
          Y_train,
          epochs=3,
          validation_data=(X_test, Y_test))

This is the error generated:

--------------------------------------------------------------------------- ValueError Traceback (most recent call last) in 65 Y_train, 66 epochs=3, ---> 67 validation_data=(X_test, Y_test))

c:\\users\\talal\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\tensorflow\\python\\keras\\engine\\training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs) 1507 steps_name='steps_per_epoch', 1508 steps=steps_per_epoch, -> 1509 validation_split=validation_split) 1510 1511 # Prepare validation data.

c:\\users\\talal\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\tensorflow\\python\\keras\\engine\\training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, batch_size, check_steps, steps_name, steps, validation_split) 991 x, y = next_element 992 x, y, sample_weights = self._standardize_weights(x, y, sample_weight, --> 993 class_weight, batch_size) 994 return x, y, sample_weights 995

c:\\users\\talal\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\tensorflow\\python\\keras\\engine\\training.py in _standardize_weights(self, x, y, sample_weight, class_weight, batch_size) 1110 feed_input_shapes, 1111
check_batch_axis=False, # Don't enforce the batch size. -> 1112 exception_prefix='input') 1113 1114 if y is not None:

c:\\users\\talal\\appdata\\local\\programs\\python\\python36\\lib\\site-packages\\tensorflow\\python\\keras\\engine\\training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix) 314 ': expected ' + names[i] + ' to have ' + 315 str(len(shape)) + ' dimensions, but got array ' --> 316 'with shape ' + str(data_shape)) 317 if not check_batch_axis: 318 data_shape = data_shape[1:]

ValueError: Error when checking input: expected lstm_18_input to have 3 dimensions, but got array with shape (366, 4)

Answer 1

your dimension is wrong for X_train.LSTM only accepts 3-dimensional inputs.you are saying that you have 4 features. Assuming that 366 is your number of timestamps for one sample, your input should be in (num_samples,366,4) shape cheers :-)

Answer 2

Your code is almost fine.

Your y_test and y_train should be an array with one element or array of shape (1,1), it doesn't matter.

Your input shape is wrong though, first LSTM should be:

model.add(LSTM(128, input_shape=(None,4), activation='relu', return_sequences=True))

Notice None , as your test and train sequences length are different, you cannot specify it (and Keras accepts first dimension unspecified). Error was due to lengths of 366 and 104 respectively. If you want to use batches with RNNs you should perform zero-padding with keras.preprocessing.sequence.pad_sequences .

No need to specify input_shape with batch, rest of the network should be fine.

And if you are performing regression, not classification, as is the case probably, you should perform two last steps written by @Ankish Bansal, eg changing loss to mean squared error and making last layer output 1 value instead of 10.

Answer 3

1. LSTM expects input of dims as (num_examples, seq_length, input_dims), so there is one error in input.

2. You are predicting output of dimension 1, what your model output is 10. Try this.

   model.add(Dense(1, activation='linear'))

3. Also, you are predicting price, that is regression problem. But, you are using classification setting. Try this

   model.compile(loss='mse', optimizer='adam', metrics=['mean_squared_error'])