Keras - Error when checking target: expected activation_5 to have shape (2,) but got array with shape (1,)

Question

In an LSTM network I'm passing as feature an array of the form

X
array([[1],[2],...,[12]],
      [[2],[3],...,[13]],...
      [[999],[1000],...,[1011]]
      [[1000],[1001],...,[1012]])

So it's shape is (1000, 12, 1)

And the target is an array with two possible values 0 and 1 of the form

y 
array([[1], [0], [0], [1], ..., [0]])

So it's shape is (1000, 1)

What I'm I doing wrong taking into account that I have to use a softmax activation and a Dense 2?

Here's the construction of the network.

model = Sequential()
model.add(LSTM(25, input_shape=(12, 1)))
model.add(Dropout(0.1))
model.add(Dense(2))
model.add(Activation('softmax'))
model.compile(loss="mse", optimizer="rmsprop")
model.fit(X, y, epochs=1000, batch_size=80, verbose=1, shuffle=False, callbacks=[EarlyStopping(patience=10)])

My guess is that it has to do with the shape of the target but I'm not sure how to fix it.

Thanks!

Answer 1

Converting target to one hot encoding with two classes can solve this problem. To convert y into one hot encoding do the following

y = numpy.eye(2)[y]

Edit:

The other solution can be changing the output layer to contain only single node with sigmoid activation. If the objective is to output values between [0, 1] sigmoid activation is best fit. I also recommend changing the loss function from 'mse' to 'binary_crossentropy', because assumption of 'mse' loss is the data is from normal distribution rather than binomial distribution. In your case the output class distribution is binomial({0, 1}). So using 'binary_crossentropy' is logical choice.

...
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss="binary_crossentropy", optimizer="rmsprop")
... 

Answer 2

I think Mitiku is right on the need to one hot encode your categorical Y. This can also be done with keras.utils.to_categorical .

one_hot_y = keras.utils.to_categorical(y)

I'm also wondering whether your model should be...

model = Sequential()
model.add(LSTM(25, input_shape=(12, 1)))
model.add(Dropout(0.1))
model.add(Dense(2, activation='softmax') # For a categorical output this has worked for me
model.compile(loss="binary_crossentropy", optimizer="rmsprop") # AFAIK 'mse' loss is not suitable for binary classification.
model.fit(X, y, epochs=1000, batch_size=80, verbose=1, shuffle=False, callbacks=[EarlyStopping(patience=10)])

I'm relatively new to Neural Networks myself so this may not fit with LSTM.