How to choose number of perceptron in fine-tuning FC layer?

Question

I use VGG-16 pre-trained model and fine-tune the last 3 FC layers. But in my case, I only use 3 classes as my classification. I want to ask how to choose the perceptron of FC layers . Should I visualize the Conv5_3 layer , then making a decision? BTW, VGG-16 official model is 4096, 4096, 1000 perceptron in FC layers.

Answer 1

The easiest way to adapt your network is to add another FC layer on top of the VGG (with weight kernel of size 1000x3).

Alternatively, replace the last FC layer (of size 4096x1000) with an FC layer of size 4096x3.

Don't forget to properly initialize your newly added layers.