Make Your Own Neural Network Back Propagation

Question

I'm reading through the Make Your Own Neural Network book and in the chapter where the author describes about the back propagation, I find myself confused. I would like to relate the way the author explains with an example where he shows a 2 node, 3 layer network sh shown in the image below:

If I construct the Matrix representation of the above Neural Network for back propagation, it would look like this:

Where W ^T _{hidden_output} is a Matrix transpose of the input weights, thus the Matrix representation in details is:

So if I want to now calculate the hidden Errors (e _{1_hidden} and e _{2_hidden} ), I have the following:

e _{1_hidden} = W ₁₁ * e ₁ + W ₁₂ * e ₂

e _{2_hidden} = W ₂₁ * e ₁ + W ₂₂ * e ₂

But if I apply the values as given in the example., where e ₁ = 1.5 and e ₂ = 0.5, I do not get the e _{1_hidden} = 0.7 and e _{2_hidden} = 1.3

Where am I going wrong with my understanding / calculation? Any help?

Answer 1

You are describing the error back propagation based on simply multiplying by link weights , whereas the picture splits the error in proportion to the link weights . Both differences are described eg on this webpage :

In the picture, you see that the error is split in proportion to the link weights , eg e ₁ = 1.5 is split into 0.6 and 0.9 according to the weights of W ₁₁ = 1.0 and W ₂₁ = 3.0. (Note further, that the subscripts of the weights are also wrong in the picture because there is only W ₁₁ and all others are denoted by W ₁₂ ...)

This split up error is then added to the final error of the hidden layer, eg:

e _{hidden 1} = 0.6 + 0.1 = 0.7.