[英]Perfect precision, recall and f1-score, yet bad prediction
Using scikit-learn to classify a binary problem. 使用scikit-learn对二进制问题进行分类。 Getting perfect
classification_report
(all 1's). 获得完美的
classification_report
(全1)。 Yet prediction gives 0.36
. 但预测得出
0.36
。 How can it be? 怎么可能?
I'm familiar with imbalanced labels. 我熟悉标签不平衡的情况。 Yet I don't think this is the case here since
f1
and the other score columns, as well as the confusion matrix, indicate perfect score. 但是我不认为这是事实,因为
f1
和其他分数列以及混淆矩阵表示完美分数。
# Set aside the last 19 rows for prediction.
X1, X_Pred, y1, y_Pred = train_test_split(X, y, test_size= 19,
shuffle = False, random_state=None)
X_train, X_test, y_train, y_test = train_test_split(X1, y1,
test_size= 0.4, stratify = y1, random_state=11)
clcv = DecisionTreeClassifier()
scorecv = cross_val_score(clcv, X1, y1, cv=StratifiedKFold(n_splits=4),
scoring= 'f1') # to balance precision/recall
clcv.fit(X1, y1)
y_predict = clcv.predict(X1)
cm = confusion_matrix(y1, y_predict)
cm_df = pd.DataFrame(cm, index = ['0','1'], columns = ['0','1'] )
print(cm_df)
print(classification_report( y1, y_predict ))
print('Prediction score:', clcv.score(X_Pred, y_Pred)) # unseen data
Output: 输出:
confusion:
0 1
0 3011 0
1 0 44
precision recall f1-score support
False 1.00 1.00 1.00 3011
True 1.00 1.00 1.00 44
micro avg 1.00 1.00 1.00 3055
macro avg 1.00 1.00 1.00 3055
weighted avg 1.00 1.00 1.00 3055
Prediction score: 0.36
The issue is that you are overfitting. 问题是您过度拟合。
There are lots of code that is not used, so let's prune: 有很多未使用的代码,所以让我们修剪一下:
# Set aside the last 19 rows for prediction.
X1, X_Pred, y1, y_Pred = train_test_split(X, y, test_size= 19,
shuffle = False, random_state=None)
clcv = DecisionTreeClassifier()
clcv.fit(X1, y1)
y_predict = clcv.predict(X1)
cm = confusion_matrix(y1, y_Pred)
cm_df = pd.DataFrame(cm, index = ['0','1'], columns = ['0','1'] )
print(cm_df)
print(classification_report( y1, y_Pred ))
print('Prediction score:', clcv.score(X_Pred, y_Pred)) # unseen data
So clearly, there is no cross validation here, and the obvious reason for a low prediction score is the overfitting of the decision tree classifier. 显然,这里没有交叉验证,而较低的预测分数的明显原因是决策树分类器的过度拟合。
Use the score from the cross validation, and you should see the issue there directly. 使用交叉验证中的分数,您应该在那里直接看到问题。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.