Getting an error while adding a dense layer in keras

Question

I was trying to implement a simple Keras cat vs dog classifier, but while adding a dense layer, it returns an value error. I'm using theano as backend . Here's the code:

from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense

classifier = Sequential()
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Flatten())
classifier.add(Dense(units = 128, activation = 'relu'))

Here's the summary of the model

While executing the last line (adding Dense layer), I'm getting the following error:

ValueError: ('The specified size contains a dimension with value <= 0', (-448, 128))

Here's my keras.json file content

{
    "backend": "theano",
    "image_data_format": "channels_first",
    "floatx": "float32",
    "epsilon": 1e-07
}

I'm not able to find the problem.

Thanks in advance!

Answer 1

You're convolving across the channels dimension, try to explicitly set the data_format parameter in convolutions and pooling like this:

classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu', data_format='channels_last'))
classifier.add(Conv2D(32, (3, 3), activation = 'relu', data_format='channels_last'))
classifier.add(MaxPooling2D(pool_size = (2, 2), data_format='channels_last'))

Or reshape your data to have shape (3, 64, 64) .

In simple terms, convolution is supposed to work roughly as shown in this gif:

You see that the gray-ish filter is strided across the pixels of your image (blue) in order to extract what are called local patterns (in green-ish). The application of this filter should ideally happen along the width and height of your image, namely the two 64-dimensions in your data.
This is also especially useful when, as it is customary, we split images in channels, usually to represent their RGB components. In this case, the same process shown in the gif is applied in parallel to the three channels, and in general can be applied to N arbitrary channels. This image should help clarify:

To cut a long story short, when you call:

 classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))

Keras by default thinks that you're passing it a 64 x 3 image with 64 channels, and tries to convolve accordingly. This is obvioulsy wrong and results in negative dimensions (note how convolutions shrink the size of the image). By specifying the 'channels_last' format, you're telling Keras how the image is oriented (with the components dimension in the last "place"), so that it is able to convolve properly across the 64 x 64 images.

Answer 2

I run your code above and the summary i get is different than yours.

You should provide more information such as the keras version and backend you use...

I suspect there is something wrong within your keras.json file

as from official keras page , check your keras.json file (located in your home directory .keras/keras.json)

It should look like

{
    "image_data_format": "channels_last",
    "image_dim_ordering": "tf",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "tensorflow"
}

or

{
    "image_data_format": "channels_last",
    "image_dim_ordering": "th",
    "epsilon": 1e-07, 
    "floatx": "float32", 
    "backend": "theano"
}