Keras LSTM: Error when checking model input dimension

Question

I am a new user of keras, and trying to implement a LSTM model. For test I declared the model like below, but it fails because of difference of input dimension. Although I found similar problems in this site, I could not find my mistakes by myself.

ValueError: 
Error when checking model input: 
expected lstm_input_4 to have 3 dimensions, but got array with shape (300, 100)

My environment

• python 3.5.2
• keras 1.2.0 (Theano)

Code

 from keras.layers import Input, Dense from keras.models import Sequential from keras.layers import LSTM from keras.optimizers import RMSprop, Adadelta from keras.layers.wrappers import TimeDistributed import numpy as np in_size = 100 out_size = 10 nb_hidden = 8 model = Sequential() model.add(LSTM(nb_hidden, name='lstm', activation='tanh', return_sequences=True, input_shape=(None, in_size))) model.add(TimeDistributed(Dense(out_size, activation='softmax'))) adadelta = Adadelta(clipnorm=1.) model.compile(optimizer=adadelta, loss='categorical_crossentropy', metrics=['accuracy']) # create dummy data data_size = 300 train = np.zeros((data_size, in_size,), dtype=np.float32) labels = np.zeros((data_size, out_size,), dtype=np.float32) model.fit(train, labels) 

Edit 1 (not working, after Marcin Możejko's comment)

Thank you Marcin Możejko. But I have a similar error like below. I updated dummy data for check. What is wrong of this code?

ValueError: Error when checking model target: expected timedistributed_36 to have 3 dimensions, but got array with shape (208, 1)

def create_dataset(X, Y, loop_back=1):
    dataX, dataY = [], []
    for i in range(len(X) - loop_back-1):
        a = X[i:(i+loop_back), :]
        dataX.append(a)
        dataY.append(Y[i+loop_back, :])
    return np.array(dataX), np.array(dataY)

data_size = 300
dataset = np.zeros((data_size, feature_size), dtype=np.float32)
dataset_labels = np.zeros((data_size, 1), dtype=np.float32)

train_size = int(data_size * 0.7)
trainX = dataset[0:train_size, :]
trainY = dataset_labels[0:train_size, :]
testX = dataset[train_size:, :]
testY = dataset_labels[train_size:, 0]
trainX, trainY = create_dataset(trainX, trainY)
print(trainX.shape, trainY.shape) # (208, 1, 1) (208, 1)

# in_size = 100
feature_size = 1
out_size = 1
nb_hidden = 8

model = Sequential()
model.add(LSTM(nb_hidden, 
               name='lstm',
               activation='tanh',
               return_sequences=True,
               input_shape=(1, feature_size)))

model.add(TimeDistributed(Dense(out_size, activation='softmax')))
adadelta = Adadelta(clipnorm=1.)
model.compile(optimizer=adadelta,
              loss='categorical_crossentropy',
              metrics=['accuracy'])
model.fit(trainX, trainY, nb_epoch=10, batch_size=1)

Answer 1

This is a really classic problem with LSTM in Keras . LSTM input shape should be 2d - with shape (sequence_length, nb_of_features) . Additional third dimension comes from examples dimension - so the table fed to model has shape (nb_of_examples, sequence_length, nb_of_features) . This is where your problem comes from. Remember that a 1-d sequence should be presented as a 2-d array with shape (sequence_length, 1) . This should be a input shape of your LSTM :

model.add(LSTM(nb_hidden, 
           name='lstm',
           activation='tanh',
           return_sequences=True,
           input_shape=(in_size, 1)))

And remember to reshape your input to appropriate format.