how to solve the conv2d error?

Question

I am a beginner in python and tensorflow. I have a error in dimention problem. Is there anyone to solve this problem? my code is as below and the error came from 'aux = Convolution2D'line. The error message is "ValueError: Negative dimension size caused by subtracting 512 from 10 for 'conv2d_15/convolution' (op: 'Conv2D') with input shapes: [?,10,10,512], [10,512,512,1].

this is tensorflow backend.

def _conv_bn_relu(nb_filter, nb_row, nb_col, subsample=(1, 1)):
        def f(input):
            conv = Convolution2D(nb_filter=nb_filter, nb_row=nb_row, nb_col=nb_col,
                                 subsample=subsample, init="he_normal",
                                 border_mode="same")(input)
            norm = BatchNormalization()(conv)
            return ELU()(norm)
        return f

def get_unet():
    inputs = Input((img_rows, img_cols, 1), name='main_input')
    conv1 = _conv_bn_relu(32, 7, 7)(inputs)
    conv1 = _conv_bn_relu(32, 3, 3)(conv1)
    pool1 = _conv_bn_relu(32, 2, 2, subsample=(2, 2))(conv1)
    drop1 = Dropout(0.5)(pool1)

    conv2 = _conv_bn_relu(64, 3, 3)(drop1)
    conv2 = _conv_bn_relu(64, 3, 3)(conv2)
    pool2 = _conv_bn_relu(64, 2, 2, subsample=(2, 2))(conv2)
    drop2 = Dropout(0.5)(pool2)

    conv3 = _conv_bn_relu(128, 3, 3)(drop2)
    conv3 = _conv_bn_relu(128, 3, 3)(conv3)
    pool3 = _conv_bn_relu(128, 2, 2, subsample=(2, 2))(conv3)
    drop3 = Dropout(0.5)(pool3)

    conv4 = _conv_bn_relu(256, 3, 3)(drop3)
    conv4 = _conv_bn_relu(256, 3, 3)(conv4)
    pool4 = _conv_bn_relu(256, 2, 2, subsample=(2, 2))(conv4)
    drop4 = Dropout(0.5)(pool4)

    conv5 = _conv_bn_relu(512, 3, 3)(drop4)
    conv5 = _conv_bn_relu(512, 3, 3)(conv5)
    drop5 = Dropout(0.5)(conv5)
    print(drop5.shape)

    # Using conv to mimic fully connected layer.
    aux = Convolution2D(nb_filter=1, nb_row=drop5._keras_shape[2], nb_col=drop5._keras_shape[3],
                        subsample=(1, 1), init="he_normal", activation='sigmoid')(drop5)
    aux = Flatten(name='aux_output')(aux)

    # up6 = concatenate([Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same')(drop5), conv4], axis=3)
    up6 = merge([UpSampling2D()(drop5), conv4], mode='concat', concat_axis=1)
    conv6 = _conv_bn_relu(256, 3, 3)(up6)
    conv6 = _conv_bn_relu(256, 3, 3)(conv6)
    drop6 = Dropout(0.5)(conv6)

    # up7 = concatenate([Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same')(drop6), conv3], axis=3)
    up7 = merge([UpSampling2D()(drop6), conv3], mode='concat', concat_axis=1)
    conv7 = _conv_bn_relu(128, 3, 3)(up7)
    conv7 = _conv_bn_relu(128, 3, 3)(conv7)
    drop7 = Dropout(0.5)(conv7)

    # up8 = concatenate([Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same')(drop7), conv2], axis=3)
    up8 = merge([UpSampling2D()(drop7), conv2], mode='concat', concat_axis=1)
    conv8 = _conv_bn_relu(64, 3, 3)(up8)
    conv8 = _conv_bn_relu(64, 3, 3)(conv8)
    drop8 = Dropout(0.5)(conv8)

    # up9 = concatenate([Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(drop8), conv1], axis=3)
    up9 = merge([UpSampling2D()(drop8), conv1], mode='concat', concat_axis=1)
    conv9 = _conv_bn_relu(32, 3, 3)(up9)
    conv9 = _conv_bn_relu(32, 3, 3)(conv9)
    drop9 = Dropout(0.5)(conv9)

    conv10 = Convolution2D(1, 1, 1, activation='sigmoid', init="he_normal", name='main_output')(drop9)

    # model = Model(inputs=[inputs], outputs=[conv10])
    model = Model(inputs=[inputs], outputs=[conv10, aux])

    # model.compile(optimizer=Adam(lr=1e-5), loss={'main_output': dice_loss},
    #               metrics={'main_output': dice},
    #               loss_weights={'main_output': 1})
    model.compile(optimizer=Adam(lr=1e-5), loss={'main_output': dice_loss, 'aux_output': 'binary_crossentropy'},
                  metrics={'main_output': dice, 'aux_output': 'acc'},
                  loss_weights={'main_output': 1, 'aux_output': 0.5})

    return model

Answer 1

I think you should change this line:

aux = Convolution2D(nb_filter=1, nb_row=drop5._keras_shape[2], nb_col=drop5._keras_shape[3],
                    subsample=(1, 1), init="he_normal", activation='sigmoid')(drop5)

to:

aux = Convolution2D(nb_filter=1, nb_row=drop5._keras_shape[1], nb_col=drop5._keras_shape[2],
                    subsample=(1, 1), init="he_normal", activation='sigmoid')(drop5)

Answer 2

I don't use Keras, but I believe the problem in your code lies within the filter size you put into

aux = Convolution2D(nb_filter=1, nb_row=drop5._keras_shape[2], nb_col=drop5._keras_shape[3], subsample=(1, 1), init="he_normal", activation='sigmoid')(drop5)

It is quite hard to deduce the dimensions of your tensors, but after having read through Keras documentation of Convolution2D , as well as having analysed the dimensions of your tensors, I assume drop5 outputs a tensor of shape (samples, new_rows, new_cols, nb_filter) ( [?,10,10,512] in your error message) . In other words, your drop5 outputs an image with dimensions 10 x 10 x 512 , or equivalently speaking 512 10 x 10 images ( this is a great read if you want to learn more about CNNs ).

When you now set nb_row=drop5._keras_shape[2] and nb_col=drop5._keras_shape[3] , you set the dimensions of the filter to nb_row=10 and nb_col=512 . This means that you will try to perform convolution on 512 10 x 10 images with a 10 x 512 shaped filter. In order to see if a filter fits an image, I would assume TensorFlow subtracts the image and filter dimensions. [10, 10] - [10, 512] = [0, -502] shows that the filter is much larger than the image, and thus convolution cannot be performed, and thus your error message.

A solution to this problem is to change your nb_row and nb_col dimensions. If you want a bigger filter size than 10 x 10 , you could resize the output image from drop5 .