ValueError: Dimensions must be equal, but are 508 and 512

Question

I am trying to create an autoencoder for 3d images and here is the model:

def create_encoder(width, height, depth):
  x = Input(shape=(height, width, depth)) 

  # Encoder
  e_conv1 = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
  pool1 = MaxPooling2D((2, 2), padding='same')(e_conv1)
  batchnorm_1 = BatchNormalization()(pool1)
  e_conv2 = Conv2D(32, (3, 3), activation='relu', padding='same')(batchnorm_1)
  pool2 = MaxPooling2D((2, 2), padding='same')(e_conv2)
  batchnorm_2 = BatchNormalization()(pool2)
  e_conv3 = Conv2D(16, (3, 3), activation='relu', padding='same')(batchnorm_2)
  h = MaxPooling2D((2, 2), padding='same')(e_conv3)

  # Decoder
  d_conv1 = Conv2D(64, (3, 3), activation='relu', padding='same')(h)
  up1 = UpSampling2D((2, 2))(d_conv1)
  d_conv2 = Conv2D(32, (3, 3), activation='relu', padding='same')(up1)
  up2 = UpSampling2D((2, 2))(d_conv2)
  d_conv3 = Conv2D(16, (3, 3), activation='relu')(up2)
  up3 = UpSampling2D((2, 2))(d_conv3)
  r = Conv2D(3, (3, 3), activation='sigmoid', padding='same')(up3)

  model = Model(x, r)
  model.compile(optimizer='adam', loss='mse')
  return model

But whenever I try to run the code:

width = 512
height = 512
depth = 3
EPOCHS = 100
BS = 128

autoencoder = create_encoder(width, height, depth)
earlystop = EarlyStopping(monitor='loss', patience=3)
H = autoencoder.fit(trainXNoisy, trainX, validation_data=(testXNoisy, testX), epochs=EPOCHS, batch_size=BS, callbacks=[earlystop])

I get this error:

Epoch 1/100
/usr/local/lib/python3.7/dist-packages/keras/optimizer_v2/adam.py:105: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  super(Adam, self).__init__(name, **kwargs)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-91-48e3c96bc5fd> in <module>()
     12 earlystop = EarlyStopping(monitor='loss', patience=3)
     13 # H = autoencoder.fit(trainXNoisy, trainX, validation_data=(testXNoisy, testX), epochs=EPOCHS, batch_size=BS, callbacks=[earlystop])
---> 14 H = autoencoder.fit(trainXNoisy, trainX, epochs=EPOCHS, batch_size=BS, callbacks=[earlystop])

1 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in autograph_handler(*args, **kwargs)
   1127           except Exception as e:  # pylint:disable=broad-except
   1128             if hasattr(e, "ag_error_metadata"):
-> 1129               raise e.ag_error_metadata.to_exception(e)
   1130             else:
   1131               raise

ValueError: in user code:

    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 878, in train_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 867, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 860, in run_step  **
        outputs = model.train_step(data)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 810, in train_step
        y, y_pred, sample_weight, regularization_losses=self.losses)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/compile_utils.py", line 201, in __call__
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 141, in __call__
        losses = call_fn(y_true, y_pred)
    File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 245, in call  **
        return ag_fn(y_true, y_pred, **self._fn_kwargs)
    File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 1204, in mean_squared_error
        return backend.mean(tf.math.squared_difference(y_pred, y_true), axis=-1)

    ValueError: Dimensions must be equal, but are 508 and 512 for '{{node mean_squared_error/SquaredDifference}} = SquaredDifference[T=DT_FLOAT](model_8/conv2d_85/Sigmoid, IteratorGetNext:1)' with input shapes: [?,508,508,3], [?,512,512,3].

I am a newbie to autoencoders (maybe to the whole machine learning world) but I tried to get information from other similar problems but I couldn't. If any one could give a better a version, it'll be way better.

Thanks,

Answer 1

The line

d_conv3 = Conv2D(16, (3, 3), activation='relu')(up2)

miss the padding argument. Therefore instead of a 256 X 256 X 16 output you get a 254 X 254 X 16, which becomes a 508 X 508 X 16 after the upsampling, and finaly a 508 X 508 X 3 after the last Conv2D

MSE Error needs to compare two images of the same size, and the input is a 512 X 512 X 3. Juste add the padding argument as you do in the other Conv2D and it should work just fine

d_conv3 = Conv2D(16, (3, 3), activation='relu', padding='same')(up2)