Pandas - pd.crosstab 给了我不正确的分类

Question

I built a classifier and wanted to try out pd.crosstab . However, it seems to give me incorrect numbers of total elements which is confusing and I can't figure out why.

Actual code:

df_confusion = pd.crosstab(pd.Series(y_pred), pd.Series(y_test), 
                           rownames=['Predicted'], colnames= ['Actual'],
                           margins=True)

Typing in jupyter notebook: df_confusion yields

**Actual**   0.0      1.0   **All**

**Predicted**   

**0.0**    6529     1951        8480

**1.0**     718     208         926

**All**     7247    2159        9406**

whereas the total number of elements of each category 0 and 1 in y_pred and y_test are as follows:

sum(y_pred==0) equals 34264
sum(y_pred==1) equals 3514

sum(y_test==1) equals 34259
sum(y_test==0) equals 3519

However importing confusion_matrix yields expected answers

from sklearn.metrics  import confusion_matrix

confusion_matrix(y_test,y_pred)
array([[34259,     0],
       [    5,  3514]], dtype=int64)

Answer 1

I had a similar problem, the total number of elements in the confusion matrix didn't match the number of elements in the series. It was caused by wrong indices. One of my series was created by dropping rows from a dataframe, while the other was created in a loop. So the indices in the 2 series didn't match. I had to apply reset_index(drop=True) on the first series.