如何将简单线性回归与梯度下降并行化-使用numpy？

Question

I'm having trouble using numpy to parallelize this for loop below (get_new_weights). With my first attempt for df_dm in update_weights, the weight is completely wrong. With my second attempt at df_dm, my weight overshoots the optimal weight.

Note - bias is a single number and weight is a single number (one variable linear regression) and X is shape (442,1) and y is shape (442,1). Also note that updating my bias term works perfectly in update_weights - its just updating the weight that I'm having trouble with.

# This is the for loop that I am trying to parallelize with numpy:
def get_new_weights(X, y, weight, bias, learning_rate=0.01):
    weight_deriv = 0
    bias_deriv = 0
    total = len(X)
    for i in range(total):
        # -2x(y - (mx + b))
        weight_deriv += -2*X[i] * (y[i] - (weight*X[i] + bias))
        # -2(y - (mx + b))
        bias_deriv += -2*(y[i] - (weight*X[i] + bias))

    weight -= (weight_deriv / total) * learning_rate
    bias -= (bias_deriv / total) * learning_rate
    return weight, bias

# This is my attempt at parallelization
def update_weights(X, y, weight, bias, lr=0.01):
    df_dm = np.average(-2*X * (y-(weight*X+bias))) # this was my first guess
    # df_dm = np.average(np.dot((-X).T, ((weight*X+bias)-y))) # this was my second guess
    df_db = np.average(-2*(y-(weight*X+bias)))
    weight = weight - (lr*df_dm)
    bias = bias - (lr*df_db)
    return weight,bias

This is the equation I am using for updating my weight and bias:

Answer 1

thanks for everyone who took a look at my question. I am loosely using the term parallelization to refer to the optimization in terms of runtime that I'm looking for by removing the need for a for loop. The answer to this problem is:

df_dm = (1/len(X)) * np.dot((-2*X).T, (y-(weight*X+bias)))

The issue here was making sure that all of the arrays resulting from the intermediate steps had the correct shape. And - for those interested in the runtime difference between these two functions: the for loop took 10 times longer.