低密度区域神经网络的回归精度

Question

I am developing a neural net which needs to predict values between -1 and 1. However, I am only really concerned about the values at the ends of scale, say between -1 and -0.7 and between 0.7 and 1.

I do not mind if 0.6, for example, gets predicted to be 0.1. However, I do want to know if it's 0.8 or 0.9.

The distribution of my data is roughly normal, so there are many more samples in the range where I'm not concerned about the accuracy. It seems therefore that the training process is likely to lead to greater accuracy in the centre.

How can I configure the training or engineer my expected result to overcome this?

Thanks very much.

Answer 1

You could assign the observations to deciles, turn it into a classification problem and either assign a greater weight to the ranges you care about in the loss or just simply oversample them during training. By default, I'd go with weighing the classes in the loss function, as it is straight-forward to match with a weighted metric. Oversampling can be useful if you know that the distribution of your training data is different from the real data distribution.

To assign certain classes a greater weight in the loss function with Keras, you can pass a class_weight parameter to Model.fit . If label 0 is the first decile and label 9 is the last decile, you could double the weight of the first and last two deciles as follows:

class_weight = {
    0: 2,
    1: 2,
    2: 1,
    3: 1,
    4: 1,
    5: 1,
    6: 1,
    7: 1,
    8: 2,
    9: 2
}
model.fit(..., class_weight=class_weight)

To oversample certain classes, you'd include them more often in the batches than the class distribution would suggest. The simplest way to implement this is to sample observation indices with numpy.random.choice that has an optional parameter to specify probabilities for each entry. (Note that Keras Model.fit also has a sample_weight parameter where you can assign weights to each observation in the training data that will be applied when computing the loss function, but the intended use case is to weigh samples by the confidence in their labels, so I don't think it's applicable here.)