Scikit学习多项式线性回归和多项式特征的系数顺序

Question

I'm fitting a simple polynomial regression model, and I want get the coefficients from the fitted model.

Given the prep code:

import pandas as pd
from itertools import product
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline

# data creation
sa = [1, 0, 1, 2, 3]
sb = [2, 1, 0, 1, 2]
raw = {'a': [], 'b': [], 'w': []}
for (ai, av), (bi, bv) in product(enumerate(sa), enumerate(sb)):
    raw['a'].append(ai)
    raw['b'].append(bi)
    raw['w'].append(av + bv)
data = pd.DataFrame(raw)

# regression
x = data[['a', 'b']].values
y = data['w']
poly = PolynomialFeatures(2)
linr = LinearRegression()
model = make_pipeline(poly, linr)
model.fit(x, y)

From this answer , I know the coefficients can obtained using with

model.steps[1][1].coef_
>>> array([  0.00000000e+00,  -5.42857143e-01,  -1.71428571e+00,
             2.85714286e-01,   1.72774835e-16,   4.28571429e-01])

But this provides a 1-dimensional array and I'm not sure which numbers correspond to which variables.

Are they ordered as a ⁰ , a ¹ , a ² , b ⁰ , b ¹ , b ² or as a ⁰ , b ⁰ , a ¹ , b ¹ , a ² , b ² ?

Answer 1

You can use the get_feature_names() of the PolynomialFeatures to know the order.

In the pipeline you can do this:

model.steps[0][1].get_feature_names()

# Output:
['1', 'x0', 'x1', 'x0^2', 'x0 x1', 'x1^2']

If you have the names of the features with you ('a', 'b' in your case), you can pass that to get actual features.

model.steps[0][1].get_feature_names(['a', 'b'])

# Output:
['1', 'a', 'b', 'a^2', 'a b', 'b^2']

Answer 2

First, the coefficients of a polynomial of degree 2 are 1, a, b, a^2, ab, and b^2 and they come in this order in the scikit-learn implementation. You can verify this by creating a simple set of inputs, eg

x = np.array([[2, 3], [2, 3], [2, 3]])
print(x)
[[2 3]
 [2 3]
 [2 3]]

And then creating the polynomial features:

poly = PolynomialFeatures(2)
x_poly = poly.fit_transform(x)
print(x_poly)
[[1. 2. 3. 4. 6. 9.]
 [1. 2. 3. 4. 6. 9.]
 [1. 2. 3. 4. 6. 9.]]

You can see that the first and second feature are a and b (without counting the bias coefficient 1), the third feature is a^2 (ie 2^2), the fourth is ab=2*3, and the last is b^2=3^2. ie you model is: