python 中 cross_val_score 的精確召回 auc

Question

python 中的 cross_val_score 可以生成各種方便的 model 性能指標。 這是我用來獲取 ROC-AUC 和 Recall 的二進制分類 model。

  import sklearn

  from sklearn.linear_model import LogisticRegression
  from sklearn.model_selection import cross_val_score
  from sklearn import metrics

  log = LogisticRegression(class_weight='balanced')

  auc = cross_val_score(log, X, y, scoring='roc_auc')
  print ("ROC-AUC (Mean->): " + str(round(100*auc.mean(), 2)) + "%" + " (Standard Deviation->): " + str(round(100*auc.std(), 2)) + "%")
    
  recall = cross_val_score(log, X, y, scoring='recall')
  print ("RECALL (Mean->): " + str(round(100*recall.mean(), 2)) + "%"+ " (Standard Deviation->): " + str(round(100*recall.std(), 2)) + "%")

對於相同的二元分類 model，如何在 cross_val_score 中包含一個用於計算精確召回 AUC 的指標？

Answer 1

我認為您應該查看 function: precision_recall_curve() ，它計算不同概率閾值的精確召回對。

嘗試以下方法：

FOLDS = 6
k_fold = KFold(n_splits=FOLDS, shuffle=True, random_state=42)

for i, (train_index, test_index) in enumerate(k_fold.split(X)):
        Xtrain, Xtest = X[train_index], X[test_index]
        ytrain, ytest = y[train_index], y[test_index]
        logistic_model.fit(Xtrain, ytrain)
        pred_proba = logistic_model.predict_proba(Xtest)
        precision, recall, _ = precision_recall_curve(ytest, pred_proba[:, 1])
        ...