python sklearn：“ sklearn.preprocessing.normalize（X，norm ='l2'）”和“ sklearn.svm.LinearSVC（penalty ='l2'）”之间有什么区别

Question

here is two method of normalize :

1:this one is using in the data Pre-Processing: sklearn.preprocessing.normalize(X, norm='l2')

2:the other method is using in the classify method : sklearn.svm.LinearSVC(penalty='l2')

i want to know ,what is the different between them? and does the two step must be used in a completely model ? is it right that just use a method is enough?

Answer 1

These 2 are different things and you normally need them both in order to make a good SVC model.

1) The first one means that in order to scale (normalize) the X data matrix you need to divide with the L2 norm of each column, which is just this : sqrt(sum(abs(X[:,j]).^2)) , where j is each column in your data matrix X . This ensures that none of the values of each column become too big, which makes it tough for some algorithms to converge.

2) Irrespective of how scaled (and small in values) your data is, there still may be outliers or some features (j) that are way too dominant and your algorithm (LinearSVC()) may over trust them while it shouldn't. This is where L2 regularization comes into play , that says apart from the function the algorithm minimizes, a cost will be applied to the coefficients so that they don't become too big . In other words the coefficients of the model become additional cost for the SVR cost function. How much cost ? is decided by the C (L2) value as C*(beta[j])^2

To sum up, first one tells with which value to divide each column of the X matrix. How much weight should a coefficient burden the cost function with is the second.