[英]L1 norm instead of L2 norm for cost function in regression model
I was wondering if there's a function in Python that would do the same job as scipy.linalg.lstsq
but uses “least absolute deviations” regression instead of “least squares” regression (OLS).我想知道 Python 中是否有一个函数可以完成与scipy.linalg.lstsq
相同的工作,但使用“最小绝对偏差”回归而不是“最小二乘”回归(OLS)。 I want to use the L1
norm, instead of the L2
norm.我想使用L1
范数,而不是L2
范数。
In fact, I have 3d points, which I want the best-fit plane of them.事实上,我有 3d 点,我想要它们中最适合的平面。 The common approach is by the least square method like this Github link .常见的方法是使用最小二乘法,如这个 Github链接。 But It's known that this doesn't give the best fit always, especially when we have interlopers in our set of data.但众所周知,这并不总是最适合的,尤其是当我们的数据集中有闯入者时。 And it's better to calculate the least absolute deviation.而且最好计算最小绝对偏差。 The difference between the two methods is explained more here . 此处更详细地解释了这两种方法之间的区别。
It'll not be solved by functions such as MAD since it's an Ax = b
matrix equations and requires loops to minimizes the results.它不会被 MAD 之类的函数解决,因为它是一个Ax = b
矩阵方程并且需要循环来最小化结果。 I want to know if anyone knows of a relevant function in Python - probably in a linear algebra package - that would calculate “least absolute deviations” regression?我想知道是否有人知道 Python 中的相关函数 - 可能在线性代数包中 - 可以计算“最小绝对偏差”回归?
This is not so difficult to roll yourself, using scipy.optimize.minimize
and a custom cost_function
.使用scipy.optimize.minimize
和自定义cost_function
自己滚动并不难。
Let us first import the necessities,让我们先进口必需品,
from scipy.optimize import minimize
import numpy as np
And define a custom cost function (and a convenience wrapper for obtaining the fitted values),并定义自定义成本函数(以及用于获取拟合值的便捷包装器),
def fit(X, params):
return X.dot(params)
def cost_function(params, X, y):
return np.sum(np.abs(y - fit(X, params)))
Then, if you have some X
(design matrix) and y
(observations), we can do the following,然后,如果您有一些X
(设计矩阵)和y
(观察值),我们可以执行以下操作,
output = minimize(cost_function, x0, args=(X, y))
y_hat = fit(X, output.x)
Where x0
is some suitable initial guess for the optimal parameters (you could take @JamesPhillips' advice here, and use the fitted parameters from an OLS approach).其中x0
是最佳参数的一些合适的初始猜测(您可以在此处接受@JamesPhillips 的建议,并使用 OLS 方法中的拟合参数)。
In any case, when test-running with a somewhat contrived example,无论如何,当用一个有点人为的例子进行测试运行时,
X = np.asarray([np.ones((100,)), np.arange(0, 100)]).T
y = 10 + 5 * np.arange(0, 100) + 25 * np.random.random((100,))
I find,我发现,
fun: 629.4950595335436
hess_inv: array([[ 9.35213468e-03, -1.66803210e-04],
[ -1.66803210e-04, 1.24831279e-05]])
jac: array([ 0.00000000e+00, -1.52587891e-05])
message: 'Optimization terminated successfully.'
nfev: 144
nit: 11
njev: 36
status: 0
success: True
x: array([ 19.71326758, 5.07035192])
And,和,
fig = plt.figure()
ax = plt.axes()
ax.plot(y, 'o', color='black')
ax.plot(y_hat, 'o', color='blue')
plt.show()
With the fitted values in blue, and the data in black.拟合值用蓝色表示,数据用黑色表示。
You can solve your problem using scipy.minimize function.您可以使用 scipy.minimize 函数解决您的问题。 You have to set the function you want to minimize (in our case a plane with the form Z= aX + bY + c) and the error function (L1 norm) then run the minimizer with some starting value.您必须设置要最小化的函数(在我们的示例中为 Z= aX + bY + c 形式的平面)和误差函数(L1 范数),然后使用某个起始值运行最小化器。
import numpy as np
import scipy.linalg
from scipy.optimize import minimize
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
def fit(X, params):
# 3d Plane Z = aX + bY + c
return X.dot(params[:2]) + params[2]
def cost_function(params, X, y):
# L1- norm
return np.sum(np.abs(y - fit(X, params)))
We generate 3d points我们生成 3d 点
# Generating 3-dim points
mean = np.array([0.0,0.0,0.0])
cov = np.array([[1.0,-0.5,0.8], [-0.5,1.1,0.0], [0.8,0.0,1.0]])
data = np.random.multivariate_normal(mean, cov, 50)
Last we run the minimizer最后我们运行最小化程序
output = minimize(cost_function, [0.5,0.5,0.5], args=(np.c_[data[:,0], data[:,1]], data[:, 2]))
y_hat = fit(np.c_[data[:,0], data[:,1]], output.x)
X,Y = np.meshgrid(np.arange(min(data[:,0]), max(data[:,0]), 0.5), np.arange(min(data[:,1]), max(data[:,1]), 0.5))
XX = X.flatten()
YY = Y.flatten()
# # evaluate it on grid
Z = output.x[0]*X + output.x[1]*Y + output.x[2]
fig = plt.figure(figsize=(10,10))
ax = fig.gca(projection='3d')
ax.plot_surface(X, Y, Z, rstride=1, cstride=1, alpha=0.2)
ax.scatter(data[:,0], data[:,1], data[:,2], c='r')
plt.show()
Note: I have used the previous response code and the code from the github as a start注意:我使用了之前的响应代码和github中的代码作为开始
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.