Incompatible shapes error while training variational autoencoder

Question

I am trying to train variational encoder. But I am getting

InvalidArgumentError: Incompatible shapes: [32,784] vs. [32,2352]

[[{{node custom_variational_layer_21/logistic_loss/mul}}]] .

I read the images using opencv and append it to the list and then I converted it into numpy array. Copied code from : http://www.stokastik.in/understanding-variational-autoencoders/
I am using convolutional variational autoencoder.

images = []

files = glob.glob('../dataset/maggi/*.*')
i=0
for file in files:
    try:
        img = cv2.imread(file)
        img = cv2.resize(img, (28,28))
        images.append(img)
    except:
        print('error')

x_train = np.asarray(images)
x_train = x_train.astype('float32') / 255.

print('Input size : ',x_train.shape)

conv_variational_autoencoder(x_train)

Output :

Input size :  (1446, 28, 28, 3)
Epoch 1/50

----------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-166-2e8711de7bdc> in <module>()
72 print('Input size : ',x_train.shape)
73 
---> 74 conv_variational_autoencoder(x_train)

<ipython-input-166-2e8711de7bdc> in conv_variational_autoencoder(X_train)
 50     adam = Adam(lr=0.0005)
 51     autoencoder.compile(optimizer=adam, loss=None)
---> 52     autoencoder.fit(X_train, shuffle=True, epochs=50, batch_size=32)
 53 
 54 

/usr/local/lib/python3.6/dist-packages/keras/engine/training.py in    fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split,   validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
1037                                         initial_epoch=initial_epoch,
1038                                         steps_per_epoch=steps_per_epoch,
-> 1039                                         validation_steps=validation_steps)
1040 
1041     def evaluate(self, x=None, y=None,

/usr/local/lib/python3.6/dist-packages/keras/engine/training_arrays.py in fit_loop(model, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch, steps_per_epoch, validation_steps)
197                     ins_batch[i] = ins_batch[i].toarray()
198 
--> 199                 outs = f(ins_batch)
200                 outs = to_list(outs)
201                 for l, o in zip(out_labels, outs):

/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py in __call__(self, inputs)
2713                 return self._legacy_call(inputs)
2714 
-> 2715             return self._call(inputs)
2716         else:
2717             if py_any(is_tensor(x) for x in inputs):

/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py in _call(self, inputs)
2673             fetched = self._callable_fn(*array_vals,     run_metadata=self.run_metadata)
2674         else:
-> 2675             fetched = self._callable_fn(*array_vals)
2676         return fetched[:len(self.outputs)]
2677 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in __call__(self, *args, **kwargs)
1437           ret = tf_session.TF_SessionRunCallable(
1438               self._session._session, self._handle, args, status,
-> 1439               run_metadata_ptr)
1440         if run_metadata:
1441           proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/errors_impl.py in __exit__(self, type_arg, value_arg, traceback_arg)
526             None, None,
527             compat.as_text(c_api.TF_Message(self.status.status)),
--> 528             c_api.TF_GetCode(self.status.status))
529     # Delete the underlying status object from memory otherwise it stays alive
530     # as there is a reference to status from this from the traceback due to

InvalidArgumentError: Incompatible shapes: [32,784] vs. [32,2352]
 [[{{node custom_variational_layer_21/logistic_loss/mul}}]]

Answer 1

Thanks for the link to the article! It's a really interesting and good writeup.

Now for the problem: As a rule: ALWAYS check your models' inputs and ouputs by using the model.summaray() function. In your case your model looks like this:

Now watch closely. Your input images are of the shape 28x28x3 like you defined yourself. But the output is 28x28x1 because the article you used trains the model on mnist, which is greyscale and thus only has 1 channel for colors, you have three.

This yields an error in the loss function, because it tries to compare how well a greyscale image looks like a color image, which of course doesn't work.

To fix this, all you have to do is go to the decoder part of the conv_variational_autoencoder(x_train) function and change the output size of the last Conv2DTranspose to be 28x28x3 instead of 28x28x1 :

#Decoder
decoder_input = Input(shape=(196,))
p = Reshape((14, 14, 1))(decoder_input)
x = Conv2DTranspose(32, (3, 3), activation='relu', padding='same')(p)
x = UpSampling2D((2, 2))(x)
# dec_out = Conv2DTranspose(1, (3, 3), activation='sigmoid', padding='same')(x)
# Change the above line to:
dec_out = Conv2DTranspose(3, (3, 3), activation='sigmoid', padding='same')(x)
decoder = Model(decoder_input, dec_out)

And it should train straight away. Good luck!