![](/img/trans.png)
[英]Why is my cross_val_score always different even when I have set my random state beforehand?
[英]Why is cross_val_score different to when I calculate it manually?
這是可重現的示例代碼:
from numpy import mean
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_validate
from sklearn.model_selection import StratifiedShuffleSplit
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import balanced_accuracy_score
# define dataset
X, y = make_classification(n_samples=1000, weights = [0.3,0.7], n_features=100, n_informative=75, random_state=0)
# define the model
model = RandomForestClassifier(n_estimators=10, random_state=0)
# evaluate the model
n_splits=10
cv = StratifiedShuffleSplit(n_splits, random_state=0)
n_scores = cross_validate(model, X, y, scoring='balanced_accuracy', cv=cv, n_jobs=-1, error_score='raise')
# report performance
print('Accuracy: %0.4f' % (mean(n_scores['test_score'])))
bal_acc_sum = []
for train_index, test_index in cv.split(X,y):
model.fit(X[train_index], y[train_index])
bal_acc_sum.append(balanced_accuracy_score(model.predict(X[test_index]),y[test_index]))
print(f"Accuracy: %0.4f" % (mean(bal_acc_sum)))
結果:
Accuracy: 0.6737
Accuracy: 0.7113
我自己計算的准確性的結果總是高於交叉驗證給我的結果。 但它應該是一樣的還是我錯過了什么? 相同的度量,相同的拆分(KFold 帶來相同的結果),相同的固定 model(其他型號表現相同),相同的隨機 state,但結果不同?
這是因為,在您的手動計算中,您已經翻轉了 balance_accuracy_score 中balanced_accuracy_score
的順序,這很重要 - 它應該是(y_true, y_pred)
( docs )。
更改此設置,您的手動計算將變為:
bal_acc_sum = []
for train_index, test_index in cv.split(X,y):
model.fit(X[train_index], y[train_index])
bal_acc_sum.append(balanced_accuracy_score(y[test_index], model.predict(X[test_index]))) # change order of arguments here
print(f"Accuracy: %0.4f" % (mean(bal_acc_sum)))
結果:
Accuracy: 0.6737
和
import numpy as np
np.all(bal_acc_sum==n_scores['test_score'])
# True
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.