如何将二进制类Logistic回归与Python合并

Question

I have a video games Dataset with many categorical columns. I binarized all these columns. Now I want to predict a column (called Rating) with Logistic Regression, but this columns is now actually binarized into 4 columns (Rating_Everyone, Rating_Everyone10+, Rating_Teen and Rating_Mature). So, I applied four times the Logistic Regression and here is my code:

df2 = pd.read_csv('../MQPI/docs/Video_Games_Sales_as_at_22_Dec_2016.csv', encoding="utf-8")
y = df2['Rating_Everyone'].values
df2 = df2.drop(['Rating_Everyone'], axis=1)
df2 = df2.drop(['Rating_Everyone10'], axis=1)
df2 = df2.drop(['Rating_Teen'], axis=1)
df2 = df2.drop(['Rating_Mature'], axis=1)
X = df2.values
Xtrain, Xtest, ytrain, ytest = train_test_split(X, y, test_size=0.20)

log_reg = LogisticRegression(penalty='l1', dual=False, C=1.0, fit_intercept=False, intercept_scaling=1,
                             class_weight=None, random_state=None, solver='liblinear', max_iter=100,
                             multi_class='ovr',
                             verbose=0, warm_start=False, n_jobs=-1)
log_reg.fit(Xtrain, ytrain)
y_val_l = log_reg.predict(Xtest)
ris = accuracy_score(ytest, y_val_l)

print("Logistic Regression Rating_Everyone accuracy: ", ris)

And again:

y = df2['Rating_Everyone10'].values
df2 = df2.drop(['Rating_Everyone'], axis=1)
df2 = df2.drop(['Rating_Everyone10'], axis=1)
df2 = df2.drop(['Rating_Teen'], axis=1)
df2 = df2.drop(['Rating_Mature'], axis=1)
X = df2.values
Xtrain, Xtest, ytrain, ytest = train_test_split(X, y, test_size=0.20)

log_reg = LogisticRegression(penalty='l1', dual=False, C=1.0, fit_intercept=False, intercept_scaling=1,
                             class_weight=None, random_state=None, solver='liblinear', max_iter=100,
                             multi_class='ovr',
                             verbose=0, warm_start=False, n_jobs=-1)
log_reg.fit(Xtrain, ytrain)
y_val_l = log_reg.predict(Xtest)
ris = accuracy_score(ytest, y_val_l)

print("Logistic Regression Rating_Everyone accuracy: ", ris)

And so on for Rating_Teen and Rating_Mature. Can you tell me how to merge all these four results into one result OR how can I do this multiclass Logistic Regression problem better?

Answer 1

The LogisticRegression model is inherently handle multiclass problems:

Below is a summary of the classifiers supported by scikit-learn grouped by strategy; you don't need the meta-estimators in this class if you're using one of these, unless you want custom multiclass

behavior: Inherently multiclass: Naive Bayes, LDA and QDA, Decision Trees, Random Forests, Nearest Neighbors, setting multi_class='multinomial' in sklearn.linear_model.LogisticRegression.

As a basic model, without class weighting (as you may need to do as samples may not be balanced over the ratings) set multi_class='multinomial' and change the solver to 'lbfgs' or one of the other solvers that support multiclass problems:

For multiclass problems, only 'newton-cg', 'sag' and 'lbfgs' handle multinomial loss; 'liblinear' is limited to one-versus-rest schemes

So you dont have to have to split your datasets up the way you have. Instead provide the original ratings column as the the labels.

Here is a minimal example:

X = np.random.randn(10, 10)
y = np.random.randint(1, 4, size=10) # 3 classes simulating ratings
lg = LogisticRegression(multi_class='multinomial', solver='lbfgs')
lg.fit(X, y)
lg.predict(X)

Edit: responding to comment.

td;lr : I expect that the model will learn that interaction on it own. IF not you might encode that information as a feature . So there is no obvious need to binarize your classes.

The way I understand it that you have features of a movies and you have the MPAA rating for the movie as the label (which you're trying to predict). This is then a multiclass problem which you can start modeling using logistic regression ( this you knew ). This is the model I proposed in above.

Now you recognized that there is a implicit distance between classes. The way I would use this information is as a feature for the model. However, I'd first be inclined to see of the model will learn this on its own.