[英]Regularization of Logistic Regression coefficients in MATLAB
I'm trying to implement a Logistic Regression with regularization (either L1 or L2). 我正在尝试使用正则化(L1或L2)实现Logistic回归。 The mnrfit() function does not implement regularization. mnrfit()函数未实现正则化。 Is there any built-in function that can do the regularization or do I have to roll my own regularization code? 是否有任何内置函数可以进行正则化,或者我必须滚动自己的正则化代码? If so, are there any tutorials that I can look at? 如果是这样,有没有我可以看的教程? The papers I have been looking at are rather mathematically dense. 我一直在看的论文在数学上相当密集。
Liblinear was the standard we used. Liblinear是我们使用的标准。
http://www.csie.ntu.edu.tw/~cjlin/liblinear/ http://www.csie.ntu.edu.tw/~cjlin/liblinear/
L1 as well as L2 regularization are very easy to implement. L1和L2正则化都很容易实现。
L1 regularization works by subtracting a fixed amount of the absolute value of your weights after each training step. L1正则化通过在每个训练步骤后减去固定的权重绝对值来进行。 So with an L1 regularization coefficient of eg 0.01, your weights (1.0, -2.0, 3.0) would become (0.99, -1.99, 2.99). 因此,如果L1正则化系数为0.01,则您的权重(1.0,-2.0,3.0)将变为(0.99,-1.99,2.99)。
L2 regularization works by subtracting a percentage of your weights. L2正则化通过减去一定百分比的权重来工作。 With a coefficient of 0.01, this means multiplying your weight vector by 1. - 0.01 = 0.99. 系数为0.01意味着将权重向量乘以1。-0.01 = 0.99。 The weights (1.0, -2.0, 3.0) would become (0.99, -1.98, 2.97). 权重(1.0,-2.0,3.0)将变为(0.99,-1.98,2.97)。 This is also known as weight decay . 这也称为重量衰减 。
As you can see, L1 regularization pulls small weights towards 0. L2 regularization on the other side has almost no effect on small weights but drastically reduces large weights. 如您所见,L1正则化将较小的权重拉向0。另一侧的L2正则化几乎对较小的权重没有影响,但会大大减少较大的权重。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.