Scikit-Learn：逻辑回归 OvR 访问二元估计器/分类器

Question

As we might know, when using solver='liblinear' on multiclass classification problem, logistic regression will use one-vs-rest strategy. Does that mean there should be n_classes number of binary classifiers/estimators? If so, how can I access those?

I have read the documentation , but could not find any way to do this.

Answer 1

It looks like there is no easy way to access those sub models. However you can recompute these sub models using model.coef_ and model.intercept_ .

As follows:

from sklearn.linear_model import LogisticRegression
from sklearn import svm, datasets
import numpy as np

X_train, y_train = datasets.load_iris(return_X_y=True)
model = LogisticRegression(
    penalty="l1",
    multi_class="ovr",
    class_weight="balanced",
    solver="liblinear",
)
model.fit(X_train, y_train)

n_labels = len(np.unique(y_train))

for i in range(n_labels):
    sub_model = LogisticRegression(penalty=model.penalty, C=model.C)
    sub_model.coef_ = model.coef_[i].reshape(1, -1)
    sub_model.intercept_ = model.intercept_[i].reshape(-1, 1)
    sub_model.classes_ = np.array([0, 1])

    y_train_ovr = np.where(y_train == i, 1, 0)
    score = sub_model.score(X_train, y_train_ovr)

    print(f"OVR for label={i}, score={score:.4f}")

Output:

OVR for label=0, score=1.0000
OVR for label=1, score=0.7333
OVR for label=2, score=0.9667

This code is basically creating a new LogisticRegression() for each label based on the original model coefficients, intercepts, C and penality. Finally the y_train labels are encoded in order to represent this OVR task.

Answer 2

The predictions of the LogisticRegression model are determined by the coef_ and intercept_ attributes estimated during fit. Look at this example:

X, y = load_iris(return_X_y=True)

# ovr model
ovr = LogisticRegression(multi_class='ovr', penalty='none').fit(X, y)

# manually estimate binary model for each label
y_0 = np.array([label == 0 for label in y])
y_1 = np.array([label == 1 for label in y])
y_2 = np.array([label == 2 for label in y])

m0 = LogisticRegression(penalty='none').fit(X, y_0)
m1 = LogisticRegression(penalty='none').fit(X, y_1)
m2 = LogisticRegression(penalty='none').fit(X, y_2)

ovr.coef_
# array([[  2.02162242,   6.99849918, -11.14813559,  -5.15488554],
#        [ -0.24535745,  -2.79656276,   1.31365383,  -2.77836927],
#        [ -2.46523384,  -6.6809256 ,   9.42922775,  18.28660819]])

m0.coef_
# array([[  2.02162242,   6.99849918, -11.14813559,  -5.15488554]])

m1.coef_
# array([[-0.24535745, -2.79656276,  1.31365383, -2.77836927]])

m2.coef_
# array([[-2.46523384, -6.6809256 ,  9.42922775, 18.28660819]])

The rows of the OVR model coefficient matrix are the coefficient vectors for the three binary problems. This is because during fitting of the OVR model, each of the three binary models is estimated. The OVR model then remembers the coef_ vectors (and also the intercept_ s).
So to answer the question: Yes, there are n_classes different binary estimators after fitting the OVR model. Each is represented by it's intercept and coefficients (but not by separate scikit-learn estimator objects).