fully connected layer size

Question

i'm using conv net for image classification I have build it from scratch

I got good result compared to litterature

network architecture:

model = Sequential()
model.add(Conv2D(24,kernel_size=3,padding='same',activation='relu',
        input_shape=(n,n,1)))
model.add(MaxPool2D())
model.add(Conv2D(48,kernel_size=3,padding='same',activation='relu'))
model.add(MaxPool2D())
model.add(Conv2D(64,kernel_size=3,padding='same',activation='relu'))
model.add(MaxPool2D(padding='same'))
model.add(Conv2D(96,kernel_size=3,padding='same',activation='relu'))
model.add(MaxPool2D(padding='same'))
model.add(Flatten())
model.add(Dense(128, activation='relu'))    # SIZE 128  FC1
model.add(Dropout(0.5))
model.add(Dense(256, activation='relu'))   # SIZE 256   FC2
model.add(Dropout(0.5))
model.add(Dense(12, activation='softmax'))
model.compile(optimizer="adam", loss="categorical_crossentropy",metrics=[recall, fmeasure,precision,"accuracy"])  

As you can see my first fully connected layer (FC1) is of size 128 and the next one is of size 256 (FC2)

Isn't it "stupid" to have a FC2 size bigger than FC1? How value on FC2 can be calculated?

Answer 1

It is not stupid or wrong, but unusual. When you are trying to make predictions, the general idea is to gradually reduce the number of neurons in fully connected layer before you reach output layer. If increasing the size is something that works for your data, then there is nothing wrong with it. Just remember that more neurons can lead to overfitting on your data too and also more computation time.

There is no rule to calculate size of FC layers. A lot of times, the first FC layer has same number of neurons as the length of the vector obtained by flattening out the last layer before the fully connected layer but this an old method and not necessary and doesnt guarantee good results.