使用逻辑回归时如何打印一个简单的特征重要性列表？

Question

I am using the dataset found here: https://www.kaggle.com/pavansubhasht/ibm-hr-analytics-attrition-dataset

My code is:

from sklearn.feature_selection import RFE
from sklearn.linear_model import LogisticRegression

log_reg_model = LogisticRegression(max_iter=1000, solver = "newton-cg")
log_reg_model = RFE(log_reg_model, 45) # using RFE to get the top 45 most important features
log_reg_model.fit(X_train_SMOTE, y_train_SMOTE) # fitting data
y_pred = log_reg_model.predict(X_test)
print("Model accruracy score: {}".format(accuracy_score(y_test, y_pred)))
print(classification_report(y_test, y_pred))

I am trying to print out the most most important features in order like when using the feature_importances_ function in Random Forest Classification.

Is the above possible using LR? I see similar questions on Stack Overflow but no answers that show the feature names and their importance.

Answer 1

To do this, you can use a method called shap , I definitely would recommend reading about SHAP before diving right into the code , as its going to be important for you and others to understand exactly what you are presenting.

However, an example of how that could work in your implementation is:

from sklearn.feature_selection import RFE
from sklearn.linear_model import LogisticRegression
import shap

log_reg_model = LogisticRegression(max_iter=1000, solver = "newton-cg")
# log_reg_model = RFE(log_reg_model, 45) # using RFE to get the top 45 most important features
log_reg_model.fit(X_train_SMOTE, y_train_SMOTE) # fitting data
y_pred = log_reg_model.predict(X_test)
print("Model accruracy score: {}".format(accuracy_score(y_test, y_pred)))
print(classification_report(y_test, y_pred))

explainer = shap.LinearExplainer(log_reg_model, X_train_SMOTE)
shap_values = explainer.shap_values(X_test[:150])

shap.summary_plot(shap_values, feature_names = X_train_SMOTE.columns)