为什么我们在代码中使用 sum 来表示偏差的梯度，为什么我们没有在代码中使用“权重”？

Question

The code of the partial derivatives of the mean square error:

w_grad = -(2 / n_samples)*(X.T.dot(y_true - y_pred))
b_grad = -(2 / n_samples)*np.sum(y_true - y_pred)

With n_samples as n, the samples number, y_true as the observations and y_pred as the predictions

My question is, why we used the sum for the gradient in the code of b ( b_grad ), and why we didn't in the code of w_grad ?

Answer 1

The original equation is:

y = Sum( W_i*x_i + B_i )

https://latex.codecogs.com/gif.latex?%5Cvec%20y%3D%5Csum_%7Bi%3D0%7D%5E%7Bn%7Dw_ix_i +b_i

If you have ten features, then you have ten W s and ten B s, and the total number of variables are twenty.

But we can just sum all B_i into one variable, and the total number of variables becomes 10+1 = 11. It is done by adding one more dimension and fixing the last x be 1. The calculation becomes below:

https://latex.codecogs.com/gif.latex?%5Cvec%20y%3D%5Cbegin%7Bbmatrix%7Dw_0%5C%5C%5Cvdots%5C%5Cw_i%5C%5Cw_b%5Cend%7Bbmatrix%7D%5Cbegin%7Bbmatrix%7Dx_0%5C%20%5Chdots%5C%20x_i%5C1%5Cend%7Bbmatrix%7D

• Someone with reputation please convert the links to image ![]URL