训练神经网络的好方法

Question

I am training a neural network model to differentiate the orange and pomegranate. In the training dataset, the background of the object (for both orange and pomegranate) is same and constant. But while testing, the background of the object is different than what I trained with.

So my first doubt is,

Is it good approach to train a model with one background (suppose white background) and test with another background (suppose grey background)?.

Second, I trained the object with different position and the same background. Since the theory says that position doesn't matter for convolution, it has ability to recognise the object placed at anywhere, because anyhow, after convolution, the dimension of the activation map decreases and the depth increases.

So my second doubt is,

Is it necessary or good approach to keep the object at different position while training the model?

Answer 1

Is it good approach to train a model with one background (suppose white background) and test with another background (suppose grey background)?.

When training a neural network, it is important to shuffle the dataset you are using and split the dataset to training and testing sets. The reason why you need to shuffle the data, is in order for your model to see all types of samples in the training set so the moment it is exposed to new unseen data, it can reflect it over the previously seen data. In the example you mentioned above, it is important to shuffle the data due to the fact that there are different background colors which can effect the prediction of the model. Therefore both the training and the testing set need to have both background colors in order for your model to give good predictions.

Is it necessary or good approach to keep the object at different position while training the model?

It is indeed better to train your model with objects in different positions due to the fact it can bring your model to predict more types of oranges or pomegranates. Note that if you are using different positions for the object you are trying to predict, it is important to have a sufficient amount of data in order for the model to give you good predictions over the testing set.

I hope this short explanation helped, if something isn't clear please let me know and I'll edit the post.

Answer 2

Is it good approach to train a model with one background (suppose white background) and test with another background (suppose grey background)?.

Background is a property of an image that is not required for distinguishing the object. You want your network to learn this behavior. Consider two cases now:

1. You give your network images with one background. Lets see what can go possibly wrong here.
   1. Assume that your background is completely black. This means that there will be 0 output for a feature map (kernel) when it was put into the background. Your network will learn that it can put any high weights for these features and it will do a good job during training as long as those weights can successfully extract feature of the classes.
   2. Now during testing, the background color is white. The same feature maps with high weight now will have very high output. These high output can saturate the non-linear unit and all categories may be classified as one category.
2. The second case where during training you shows images with different background.
   1. In this case, neural network has to learn that the feature maps corresponding to background and need to subtract the bias based on the background.

In short, there is an extra information that you need to learn that is background is not important for deciding the category. When you provide only one color background, your neural network cannot learn this behavior and can give garbage result on test dataset.

Is it necessary or good approach to keep the object at different position while training the model?

You are right, Convolutional Neural Network are translational-equivariant. But for building a classifier, you pass the output of CNN-layer through a fully-connected layer. If you put image at different positions, different input will go to the fully-connected layer but output for all these images is the same category. So you are forcing your neural network to learn that the position of the object is not required for classifying its category.

Answer 3

Regarding your first doubt, It is not much of an issue as long as the target object is present in the images. Shuffle the data before feeding it to the network.

For second doubt, Yes it is always a good idea have target object at different positions. Also one more thing to take care is that the source of your data is same and mostly of same quality. Otherwise performance issue will arise.