[英]How can I use k-fold cross-validation in scikit-learn to get precision-recall per fold?
Let's say I have this scenario: 假设我有这种情况:
from sklearn import model_selection
from sklearn.linear_model import LogisticRegression
kfold = model_selection.KFold(n_splits=5, random_state=7)
acc_per_fold = model_selection.cross_val_score(LogisticRegression(),
x_inputs, np.ravel(y_response), cv=kfold, scoring='accuracy')
What else can I get from model_selection.cross_val_score()
? 我还能从
model_selection.cross_val_score()
得到什么? Is there a way to see what happens inside every actual fold? 有没有办法查看每个实际折痕内部发生的情况? Can I get precision-recall per fold?
我可以得到每折的精确召回率吗? Predicted values?
预测值? How about using a trained model from a fold to make predictions on unseen data?
如何使用训练有素的模型来对看不见的数据进行预测?
You can use the cross_validate
function to see what happens in each fold. 您可以使用
cross_validate
函数查看每折的情况。
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_validate
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix, recall_score, roc_auc_score, precision_score
X, y = make_classification(
n_classes=2, class_sep=1.5, weights=[0.9, 0.1],
n_features=20, n_samples=1000, random_state=10
)
clf = LogisticRegression(class_weight="balanced")
scoring = {'accuracy': 'accuracy',
'recall': 'recall',
'precision': 'precision',
'roc_auc': 'roc_auc'}
cross_val_scores = cross_validate(clf, X, y, cv=3, scoring=scoring)
The output is the following, 输出如下:
{'fit_time': array([ 0. , 0. , 0.01559997]),
'score_time': array([ 0.01559997, 0. , 0. ]),
'test_accuracy': array([ 0.9251497 , 0.95808383, 0.93674699]),
'test_precision': array([ 0.59183673, 0.70833333, 0.63636364]),
'test_recall': array([ 0.85294118, 1. , 0.84848485]),
'test_roc_auc': array([ 0.96401961, 0.99343137, 0.96787271]),
'train_accuracy': array([ 0.96096096, 0.93693694, 0.95209581]),
'train_precision': array([ 0.73033708, 0.62376238, 0.69148936]),
'train_recall': array([ 0.97014925, 0.94029851, 0.95588235]),
'train_roc_auc': array([ 0.99426906, 0.98509954, 0.99223039])}
So what happened in the first fold ? 那么第一折发生了什么?
FOLD, METRIC = (0, 'test_precision')
cross_val_scores[METRIC][FOLD]
And is the precision score
stable ? precision score
稳定?
np.std(cross_val_scores[METRIC])
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.