[英]Python Numpy Logistic Regression
I'm trying to implement vectorized logistic regression in python using numpy. 我正在尝试使用numpy在python中实现矢量化逻辑回归。 My Cost function (CF) seems to work OK.
我的成本函数(CF)似乎可以正常工作。 However there is a problem with gradient calculation.
但是,梯度计算存在问题。 It returns 3x100 array whereas it should return 3x1.
它返回3x100数组,而应该返回3x1。 I think there is a problem with the
(hypo-y)
part. 我认为
(hypo-y)
部分有问题。
def sigmoid(a):
return 1/(1+np.exp(-a))
def CF(theta,X,y):
m=len(y)
hypo=sigmoid(np.matmul(X,theta))
J=(-1./m)*((np.matmul(y.T,np.log(hypo)))+(np.matmul((1-y).T,np.log(1-hypo))))
return(J)
def gr(theta,X,y):
m=len(y)
hypo=sigmoid(np.matmul(X,theta))
grad=(1/m)*(np.matmul(X.T,(hypo-y)))
return(grad)
X
is a 100x3 arrray, y
is 100x1, and theta
is a 3x1 arrray. X
是100x3的错误, y
是100x1, theta
是3x1的错误。 It seems both functions are working individually, however this optimization function gives an error: 似乎这两个函数都在单独工作,但是此优化函数给出了一个错误:
optim = minimize(CF, theta, method='BFGS', jac=gr, args=(X,y))
The error: "ValueError: shapes (3,100) and (3,100) not aligned: 100 (dim 1) != 3 (dim 0)"
错误:“ ValueError:形状(3,100)和(3,100)不对齐:100(dim 1)!= 3(dim 0)”
I think there is a problem with the (hypo-y) part.
我认为(hypo-y)部分有问题。
Spot on! 发现!
hypo
is of shape (100,)
and y
is of shape (100, 1)
. hypo
的形状为(100,)
, y
的形状为(100, 1)
。 In the element-wise -
operation, hypo
is broadcasted to shape (1, 100)
according to numpy's broadcasting rules . 在逐元素
-
的操作, hypo
被广播到形状(1, 100)
根据numpy的的广播规则 。 This results in a (100, 100)
array, which causes the matrix multiplication to result in a (3, 100)
array. 这导致一个
(100, 100)
数组,这导致矩阵相乘得到一个(3, 100)
数组。
Fix this by bringing hypo
into the same shape as y
: 通过使
hypo
具有与y
相同的形状来解决此问题:
hypo = sigmoid(np.matmul(X, theta)).reshape(-1, 1) # -1 means automatic size on first dimension
There is one more issue: scipy.optimize.minimize
(which I assume you are using) expects the gradient to be an array of shape (k,)
but the function gr
returns a vector of shape (k, 1)
. 还有一个问题:
scipy.optimize.minimize
(我假设您正在使用)期望渐变是形状为(k,)
的数组(k,)
但是函数gr
返回形状为(k, 1)
的向量。 This is easy to fix: 这很容易解决:
return grad.reshape(-1)
The final function becomes 最终功能变为
def gr(theta,X,y):
m=len(y)
hypo=sigmoid(np.matmul(X,theta)).reshape(-1, 1)
grad=(1/m)*(np.matmul(X.T,(hypo-y)))
return grad.reshape(-1)
and running it with toy data works (I have not checked the math or the plausibility of the results): 并使用玩具数据运行它(我尚未检查结果的数学或合理性):
theta = np.reshape([1, 2, 3], 3, 1)
X = np.random.randn(100, 3)
y = np.round(np.random.rand(100, 1))
optim = minimize(CF, theta, method='BFGS', jac=gr, args=(X,y))
print(optim)
# fun: 0.6830931976615066
# hess_inv: array([[ 4.51307367, -0.13048255, 0.9400538 ],
# [-0.13048255, 3.53320257, 0.32364498],
# [ 0.9400538 , 0.32364498, 5.08740428]])
# jac: array([ -9.20709950e-07, 3.34459058e-08, 2.21354905e-07])
# message: 'Optimization terminated successfully.'
# nfev: 15
# nit: 13
# njev: 15
# status: 0
# success: True
# x: array([-0.07794477, 0.14840167, 0.24572182])
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.