PyTorch 如何仅用标量损失训练 NN？

Question

Let say we have a NN we want to train to predict 3 values from an input. We have a set of training data:

x_train = ((1, 5, 3, 2, 6), (1, 8, 6, 9, 3), ...) 

and the targets

y_train = ((25, 32, 0.12), (.125, -5, 8), ...)

How can pytorch do the training if it just computes a scalar as a loss function? Why it is not able to compute the loss associated with each output neuron? For example if the answer to x_train[0] is (20, 32, 0.12), we dont want want to update the same weights as of the answer was (25, 37, 0.12), right? But in that case, the loss computed with pytorch would be the same, as it would (for the classical MSE loss) mean all errors.

How pytorch can train correclty a nn without knowing from where the error comes from?

Answer 1

Your question is not related to pytorch but to general NN training.

In the end every training needs to minimize the loss. What will be the loss can be many things but in the end it has to be scalar becasue minimize a vector is ambigious.

The NN optimizer not only calculates the loss, but also calculates the gradient of this loss with respect to all the trainable parameters of the NN. So in hand waving (without getting into tons of methods), parameters that has large effect of the loss will move a lot and parameters that hardly affect the loss will hardly change.

This way it kind of know "from where the error comes from"