邏輯回歸 Model 使用正則化 (L1 / L2) Lasso 和 Ridge

Question

我正在嘗試構建 model 並創建網格搜索，下面是代碼。 原始數據是從該站點下載的（信用卡欺詐數據）。 https://www.kaggle.com/mlg-ulb/creditcardfraud

讀取數據后從標准化開始的代碼。

standardization = StandardScaler()
credit_card_fraud_df[['Amount']] = standardization.fit_transform(credit_card_fraud_df[['Amount']])
# Assigning feature variable to X
X = credit_card_fraud_df.drop(['Class'], axis=1)

# Assigning response variable to y
y = credit_card_fraud_df['Class']
# Splitting the data into train and test
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.7, test_size=0.3, random_state=100)
X_train.head()
power_transformer = PowerTransformer(copy=False)
power_transformer.fit(X_train)                       ## Fit the PT on training data
X_train_pt_df = power_transformer.transform(X_train)    ## Then apply on all data
X_test_pt_df = power_transformer.transform(X_test)
y_train_pt_df = y_train
y_test_pt_df = y_test
train_pt_df = pd.DataFrame(data=X_train_pt_df, columns=X_train.columns.tolist())
# set up cross validation scheme
folds = StratifiedKFold(n_splits = 5, shuffle = True, random_state = 4)

# specify range of hyperparameters
params = {"C":np.logspace(-3,3,5,7), "penalty":["l1","l2"]}# l1 lasso l2 ridge

## using Logistic regression for class imbalance
model = LogisticRegression(class_weight='balanced')
grid_search_cv = GridSearchCV(estimator = model, param_grid = params, 
                        scoring= 'roc_auc', 
                        cv = folds, 
                        return_train_score=True, verbose = 1)            
grid_search_cv.fit(X_train_pt_df, y_train_pt_df)
## reviewing the results
cv_results = pd.DataFrame(grid_search_cv.cv_results_)
cv_results

示例結果：

  mean_fit_time std_fit_time    mean_score_time std_score_time  param_C param_penalty   params  split0_test_score   split1_test_score   split2_test_score   split3_test_score   split4_test_score   mean_test_score std_test_score  rank_test_score
    0   0.044332    0.002040    0.000000    0.000000    0.001   l1  {'C': 0.001, 'penalty': 'l1'}   NaN NaN NaN NaN NaN NaN NaN 6
    1   0.477965    0.046651    0.016745    0.003813    0.001   l2  {'C': 0.001, 'penalty': 'l2'}   0.485714    0.428571    0.542857    0.485714    0.457143    0.480000    0.037904    5

我在輸入數據中沒有任何 null 值。我不明白為什么我得到這些列的 Nan 值。 誰能幫幫我嗎？

Answer 1

您在此處定義的默認求解器有問題：

model = LogisticRegression(class_weight='balanced')

這是從以下錯誤消息中得出的：

ValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty.

此外，在定義參數網格之前研究文檔可能會有用：

penalty: {'l1', 'l2', 'elasti.net', 'none'}, default='l2' 用於指定懲罰中使用的范數。 “newton-cg”、“sag”和“lbfgs”求解器僅支持 l2 懲罰。 “elasti.net”僅受“saga”求解器支持。 如果為“無”（liblinear 求解器不支持），則不應用正則化。

一旦您使用支持所需網格的不同求解器更正它，您就可以達到 go：

## using Logistic regression for class imbalance
model = LogisticRegression(class_weight='balanced', solver='saga')
grid_search_cv = GridSearchCV(estimator = model, param_grid = params, 
                        scoring= 'roc_auc', 
                        cv = folds, 
                        return_train_score=True, verbose = 1)            
grid_search_cv.fit(X_train_pt_df, y_train_pt_df)
## reviewing the results
cv_results = pd.DataFrame(grid_search_cv.cv_results_)

還要注意ConvergenceWarning ，它可能建議您需要增加默認max_iter 、 tol或切換到另一個求解器並重新考慮所需的參數網格。