将系数分配回多元线性回归中的分类变量

Question

From running a multiple linear regression using Sciki-learn, I need to obtain a equation like Y= a + bX1 + cX2 + dX2 + eX3 + fX4 + gX5 where b, c, d, e, f and g are coefficients of each of the independent variables.

I have performed multiple linear regression using Scikit-learn with 3 categorical variables (Cat V) and 2 continuous variables (Cont V) as below

    Cat V 1    Cat V 2    Cat V 3    Cont V 1    Cont V 2
    A          C3         X2         208         3000
    B          C6         X4         256         4000
    B          C7         X5         275         2000
    C          C2         X1         508         3200

I have encoded the categorical data using column transformer, which has resulted in many more columns, as each categorical variable has more than 10 different categories. The code I have used to perform this is below

    # Encoding categorical data
    mct = make_column_transformer((OneHotEncoder(drop='first'), [0, 1, 2]), remainder = 'passthrough')
    X = mct.fit_transform(X)

    # Splitting the dataset into the Training set and Test set
    X_train, X_test, y_train, y_test = train_test_split(X, y, 
    test_size=0.2, random_state = 0)

    # Fitting Multiple Linear Regression to the Training set
    regressor = LinearRegression()
    regressor.fit(X_train, y_train)

I have found the coefficients of each variable (after encoding) using the the [.coef_] function with following code

    print(regressor.coef_)

The problem is this show coefficients of the variables after being split in encoding, like below

    [ 1.80198679e-05 -5.55304459e-05  1.90462615e-03 -6.22320276e-05
  1.17184589e-03  .... -2.33744077e-03 -1.91538011e-04
  8.61626216e-11  3.73358813e-03]

I need to find the 5 coefficients of the original 5 variables. like

    Cat V 1     Coefficient 1
    Cat V 2     Coefficient 2
    Cat V 3     Coefficient 3
    Cont V 1    Coefficient 4
    Cont V 2    Coefficient 5

Is it possible to do this?

Answer 1

Linear regression means you're searching f in y=f(x), or y=f(x1,x2..) for continuous variables. The machinery does not work for categories: it thinks that a variable corresponding to a category can smoothly vary between C2 and C3, C3 and C4, etc. When you created several columns, maybe things got worse: now you have more variables that try to accomodate the shape of f() - see what I mean? Think of a single column of categories, y=f(c); now you have y=f(c1,c2...), each continuously varying and, this way, mixing categories together in small amounts (your coefficients, as 10^-5, 10^-6, etc.).

Logistic regression employs af() with a curious shape (the sigmoid) with extreme values 0 and 1 and a ramp in between; it is continuous between Cx and Cy but has a sudden jump. It is often associated with this type of problem. Neural Networks as multi-layer perceptron are nothing but regression decorated with fancy names as AI, neural, etc. Does it solve your problem? It depends - period. But dozens of papers were published by running such a regression, tweaking parameters and 'learning' algorithms and tagging the whole thing with hot-topic-words.

If - and only if - there's some logic in the idea of transitioning from one category to another (suppose an object might be in an intermediary state), you might code your categories as numbers. Maybe C1=1, C2=2, etc. At the end, the continuous values might be an indication of a variable approximately matching a category - or none of that, simply the variable was distorted enough to make f() fit the best the outputs y1,y2.. you provided. See, there's no definitive answer here. Any way you do it, is approximative.

Instead of using linear regression you might try to fit another curve (eg parabolic, sin..), but that brings a bunch of new problems. The MLP (perceptron) is a summation of sigmoids and has nice approximmation capabilities (compared with parabola, sin...), hence the interest on it.

Then, there's SVM (Support Vector Machine), another beast in the scene; same basic idea, but you work with something as y=f(g(x)) for some crazy g() which makes it easy to find f().

Another shot, things as Tree Decision Learning and Case Based Reasoning; this can be performed with tools as RapidMiner with weka plugin, or weka itself.

Simple linear regression is a complicate problem - not because of the math (which can be presented in horrible ways), but because of the subtleties around the data and how it represents something in the real world. And.. you have something more difficult than simple linear regression (sorry for the bad news). Hope you find an acceptable solution.