这些是卷积神经网络的可取的最佳实践吗？

Question

This is my understanding of convolutional neural networks and my best practices. Please advise if my though-process is accurate.

1) If input set has small details, have a smaller filter size.

2) Qty is dependent on how many small details.

3) Always have a maxpool layer for showing the output layers the most important parts of the image.

4) if accuracy between the last few epochs is very small, and training time is long, consider increasing learning rate and/or decreasing epochs

5) if dropout is to be used, use it between two dense layers, preferably right before the output layer

Thanks for your expertise in advance.

Answer 1

Let's evaluate point by point.

1) If input set has small details, have a smaller filter size. -> Yes, you can use the most 3x3 common filter or if need be then 2x2. But, still you will have to go through a lot of prototyping. An added fact to this, the number of filters is directly proportional to the position of layers. So, in first layer, have 16, then next layers 32 and so on.

2) Qty is dependent on how many small details. -> You don't know the number of small details. Neural networks are effectively a black-box, there is no rigorous mathematical way to define neural networks till now.

3) Always have a maxpool layer for showing the output layers the most important parts of the image. -> By that logic, you would just put MaxPool at the end of the network. Usually, have a MaxPool, after 2-3 Conv layers. The point of MaxPool is to highlight the important features but also at the same time to reduce parameter space. Using it sparingly defeats it purpose.

4) If accuracy between the last few epochs is very small, and training time is long, consider increasing learning rate and/or decreasing epochs. -> Accuracy usually slows increases towards the end of training. Even in learning rate schedules, you decrease the learning rate w.r.t to increasing epochs. Don't expect big jumps in accuracy towards the end of training. Try to find a good learning rate and train till satisfaction w.r.t validation set.

5) If dropout is to be used, use it between two dense layers, preferably right before the output layer. -> The point of dropout is to give a regularization effect. You can use dropout in Conv layers also. Use dropout to reduce overfitting, the place doesn't matter. The effectiveness of using dropout in Dense layers should be more as they are towards the end of network holding the weights that result into your prediction but dropout should be used throughout the network if possible. Or you can use regularizer on layer weights.