Different Input/Output shape in Keras

Question

I'm new to all this Neural Networks thing and I'm actually trying some toy codes with different codig options (raw Python, TF...)

Currently, I've made a simple binary AND, OR and NOT operator solving network in TFLearn:

# 1. Import library of functions
import numpy as np
import tflearn
from keras.models import Sequential
from keras.layers import Dense, Activation

# 2. Logical data
input = [[0., 0.], [0., 1.], [1., 0.], [1., 1.]]
YOR = [[0.], [1.], [1.], [1.]]
YAND=[[0.], [0.], [0.], [1.]]
YNOT=[[0.], [1.], [1.], [0.]]

######   VERSION TFLEARN     #####
# 3. Building our neural network/layers of functions 
neural_net = tflearn.input_data(shape=[None, 2])
neural_net = tflearn.fully_connected(neural_net, 1, activation='sigmoid')
neural_net = tflearn.regression(neural_net, optimizer='sgd', learning_rate=2, loss='mean_square')

# 4. Train the neural network / Epochs
model = tflearn.DNN(neural_net,tensorboard_verbose=0)
model.fit(input, YOR, n_epoch=1000, snapshot_epoch=False)

# 5. Testing final prediction
print("Testing OR operator")
print("0 or 0:", model.predict([[0., 0.]]))
print("0 or 1:", model.predict([[0., 1.]]))
print("1 or 0:", model.predict([[1., 0.]]))
print("1 or 1:", model.predict([[1., 1.]]))

Now I'm trying to replicate it in Keras (using CNTK backend) using this code:

# 2. Logical OR operator / the data
input = np.array([[0., 0.], [0., 1.], [1., 0.], [1., 1.]])
YOR = np.array([[0.], [1.], [1.], [1.]])
YAND=np.array([[0.], [0.], [0.], [1.]])
YNOT=np.array([[0.], [1.], [1.], [0.]])

######   VERSION KERAS     #####
# 3. Building our neural network/layers of functions 
model= Sequential()
model.add(Dense(4,input_shape=[2,]))
model.add(Activation('sigmoid'))

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy'])

# 4. Train the neural network / Epochs
model.fit(input,YOR,epochs=1000,verbose=1)

# 5. Testing final prediction
print("Testing OR operator")
print("0 or 0:", model.predict([[0., 0.]]))
print("0 or 1:", model.predict([[0., 1.]]))
print("1 or 0:", model.predict([[1., 0.]]))
print("1 or 1:", model.predict([[1., 1.]]))

On execution, I would expect to obtain the result of the operator in each case, but instead I got the following error:

ValueError: Error when checking input: expected dense_1_input to have shape (2,) but got array with shape (1,)

According to Keras Doc , seems to be that the output shape must be the same as the input shape, and though I can modify the input_shape, apparently doesn't recognize the output_shape arg.

By the way if I try to change the value of the input_shape in order to fit it to the output (according to what i just mention) I get the same message but swapping those values.

Does this mean that I can only obtain results of the same shape as the input?

Answer 1

I tried running the program you have given. But it produced a different type of error for me

Error when checking target: expected activation_13 to have shape (4,) but got array with shape (1,)

I changed value inside Dense to solve the above error. Why don't you try using this

model= Sequential()
model.add(Dense(1,input_shape=(2,)))
model.add(Activation('sigmoid'))

model.compile(optimizer='rmsprop',
          loss='binary_crossentropy',
          metrics=['accuracy'])

# 4. Train the neural network / Epochs
model.fit(input,YOR,epochs=1000,verbose=1)

# 5. Testing final prediction
print("Testing OR operator")
test = np.array([[0., 0.]])
print("0 or 0:", model.predict(test))
test = np.array([[0., 1.]])
print("0 or 1:", model. model.predict(test))
test = np.array([[1., 0.]])
print("1 or 0:",  model.predict(test))
test = np.array([[1., 1.]])
 print("1 or 1:",  model.predict(test))

Also we can train models in Keras even if the input and output shape are different

Answer 2

I want to add something to the already given answer. Because you actually can keep the line as it was with 4 units resp. hidden size:

model.add(Dense(4, input_shape=(2,))) 

So assuming you want to keep a hidden size of 4 , then you need to add just an proper output layer where the shape is matching the shape of your data.

In you case:

model.add(Dense(1))

So if you want to keep the hidden size different than 1 this is probably what you want, here is the full working code:

Note: I also added another activation for the output layer.

import numpy as np

from keras.models import Sequential
from keras.layers import Dense, Activation

# 2. Logical OR operator / the data
input = np.array([[0., 0.], [0., 1.], [1., 0.], [1., 1.]])
YOR = np.array([[0.], [1.], [1.], [1.]])
YAND=np.array([[0.], [0.], [0.], [1.]])
YNOT=np.array([[0.], [1.], [1.], [0.]])

######   VERSION KERAS     #####
# 3. Building our neural network/layers of functions 
model= Sequential()
model.add(Dense(4, input_shape=(2,)))
# you can place 
model.add(Activation('sigmoid'))
# layer to match output shape
model.add(Dense(1))
# of course you can add a sigmoid or other 
# activation here to match you target value range
model.add(Activation('sigmoid'))

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy'])

# 4. Train the neural network / Epochs
print(input.shape)
model.fit(input,YOR,epochs=1000,verbose=1)

# 5. Testing final prediction
print("Testing OR operator")
print("0 or 0:", model.predict([[0., 0.]]))
print("0 or 1:", model.predict([[0., 1.]]))
print("1 or 0:", model.predict([[1., 0.]]))
print("1 or 1:", model.predict([[1., 1.]]))

I hope this makes things clearer and helps you to understand the error message better.