具有分类特征的线性回归分析

Question

Regression algorithms working fine on represented as numbers. It's quite clear how to do regression on data which contains numbers and predict output. However I need to do regression analysis on data that contains categorical feature. I have a csv file which contains two columns install-id and page-name both are object type. I need to give install-id as input and page-name should be predicted as output. Below is my code. Please help me in this.

import pandas as pd
data = pd.read_csv("/Users/kashifjilani/Downloads/csv/newjsoncontent.csv")
X = data["install-id"]
Y = data["endPoint"]
X = pd.get_dummies(data=X, drop_first=True)
from sklearn import linear_model
from sklearn.model_selection import train_test_split
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = .20, random_state = 40)
regr = linear_model.LinearRegression()
regr.fit(X_train, Y_train)
predicted = regr.predict(X_test)

Answer 1

For the demonstration, let's say, you have this dataframe where IQ and Gender are input features. Target variable is Test Score .

|   Student |   IQ | Gender   |   Test Score |
|----------:|-----:|:---------|-------------:|
|         1 |  125 | Male     |           93 |
|         2 |  120 | Female   |           86 |
|         3 |  115 | Male     |           96 |
|         4 |  110 | Female   |           81 |
|         5 |  105 | Male     |           92 |
|         6 |  100 | Female   |           75 |
|         7 |   95 | Male     |           84 |
|         8 |   90 | Female   |           77 |
|         9 |   85 | Male     |           73 |
|        10 |   80 | Female   |           74 |

Here, IQ is numerical and Gender is a categorical feature. In the preprocessing step, we'll apply simple imputer on the numerical and one-hot-encoder on the categorical feature. You can use sklearn's Pipeline & ColumnTransformer feature for that. Then you can use your model of choice to train and predict easily.

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn import linear_model

# defining the data
d = {
    "Student": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
    "IQ": [125, 120, 115, 110, 105, 100, 95, 90, 85, 80,],
    "Gender": [
        "Male",
        "Female",
        "Male",
        "Female",
        "Male",
        "Female",
        "Male",
        "Female",
        "Male",
        "Female",
    ],
    "Test Score": [93, 86, 96, 81, 92, 75, 84, 77, 73, 74],
}

# converting into pandas dataframe
df = pd.DataFrame(d)

# setting the student id as index to keep track
df = df.set_index("Student")

# column transformation
categorical_columns = ["Gender"]
numerical_columns = ["IQ"]

# determine X
X = df[categorical_columns + numerical_columns]
y = df["Test Score"]

# train test split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, random_state=42, test_size=0.3
)

# categorical pipeline
categorical_pipe = Pipeline([("onehot", OneHotEncoder(handle_unknown="ignore"))])

# numerical pipeline
numerical_pipe = Pipeline([("imputer", SimpleImputer(strategy="mean")),])

# aggregating both the pipeline
preprocessing = ColumnTransformer(
    [
        ("cat", categorical_pipe, categorical_columns),
        ("num", numerical_pipe, numerical_columns),
    ]
)


rf = Pipeline(
    [("preprocess", preprocessing), ("classifier", linear_model.LinearRegression())]
)

# train
rf.fit(X_train, y_train)

# predict
predict = rf.predict(X_test)

This shows,

>> array([84.48275862, 84.55172414, 79.13793103])

Answer 2

I think here we have to remember the assumptions of Regression model. since we are trying to predict/identify trend between Independent Variable(X) and dependent variable(y). - linearity separable - Independent variable have restricted multicollinearity. - Homoscedasticity

As given in your example you have only one independent variable, and to summarize trend between X and y both should be linear.

suppose for an example you have given a task to predict Total-Travel time of the trip. and your data-set has following variables IV - Miles Traveled, NoOfDeliveries, GasPrice and City DV - Traveltime

Here you can see it is a mixture of Numerical(Miles Traveled, GasPrice) + categorical variables(NoOfDeliveries, City). now you have to encode these categorical variables to numbers (in order to work with regression analysis) and predict the output.

To encode the categorical variables to Binary format we are using 2 objects from sklearn library here - LabelEncoder and OneHotEncoder.

Please follow the below links to know more about how to deal with Categorical variables

Please find the below link to know more about Dummy variable Trap

Please find the below link to know more about building simple Linear regression model