[英]How to find 2 parameters with gradient descent method in Python?
I have a few lines of code which doesn't converge.我有几行不收敛的代码。 If anyone has an idea why, I would greatly appreciate.
如果有人知道为什么,我将不胜感激。 The original equation is written in def f(x,y,b,m) and I need to find parameters b,m.
原始方程写在 def f(x,y,b,m) 中,我需要找到参数 b,m。
np.random.seed(42)
x = np.random.normal(0, 5, 100)
y = 50 + 2 * x + np.random.normal(0, 2, len(x))
def f(x, y, b, m):
return (1/len(x))*np.sum((y - (b + m*x))**2) # it is supposed to be a sum operator
def dfb(x, y, b, m): # partial derivative with respect to b
return b - m*np.mean(x)+np.mean(y)
def dfm(x, y, b, m): # partial derivative with respect to m
return np.sum(x*y - b*x - m*x**2)
b0 = np.mean(y)
m0 = 0
alpha = 0.0001
beta = 0.0001
epsilon = 0.01
while True:
b = b0 - alpha * dfb(x, y, b0, m0)
m = m0 - alpha * dfm(x, y, b0, m0)
if np.sum(np.abs(m-m0)) <= epsilon and np.sum(np.abs(b-b0)) <= epsilon:
break
else:
m0 = m
b0 = b
print(m, f(x, y, b, m))
Both derivatives got some signs mixed up:两种衍生品都有一些混淆的迹象:
def dfb(x, y, b, m): # partial derivative with respect to b
# return b - m*np.mean(x)+np.mean(y)
# ^-------------^------ these are incorrect
return b + m*np.mean(x) - np.mean(y)
def dfm(x, y, b, m): # partial derivative with respect to m
# v------ this should be negative
return -np.sum(x*y - b*x - m*x**2)
In fact, these derivatives are still missing some constants:事实上,这些导数仍然缺少一些常数:
dfb
should be multiplied by 2
dfb
应乘以2
dfm
should be multiplied by 2/len(x)
dfm
应乘以2/len(x)
I imagine that's not too bad because the gradient is scaled by alpha
anyway, but it could make the speed of convergence worse.我想这还不错,因为无论如何梯度都会按
alpha
进行缩放,但这可能会使收敛速度变差。
If you do use the correct derivatives, your code will converge after one iteration:如果您确实使用了正确的导数,您的代码将在一次迭代后收敛:
def dfb(x, y, b, m): # partial derivative with respect to b
return 2 * (b + m * np.mean(x) - np.mean(y))
def dfm(x, y, b, m): # partial derivative with respect to m
# Used `mean` here since (2/len(x)) * np.sum(...)
# is the same as 2 * np.mean(...)
return -2 * np.mean(x * y - b * x - m * x**2)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.