Display misclassified digits from confusion matrix

Question

I wrote a function to find the confusion matrix of my model:

NN_model = KNeighborsClassifier(n_neighbors=1)
NN_model.fit(mini_train_data, mini_train_labels)
# Create the confusion matrix for the dev data
confusion = confusion_matrix(dev_labels, NN_model.predict(dev_data))
print(confusion)

But I am having trouble in displaying images of above 5 digits that are often confused with others. But when I try the following code, I am not getting the expected results.

index = 0
misclassifiedIndexes = []
for label, predict in zip(dev_labels, predictions):
     if label != predict: 
        misclassifiedIndexes.append(index)
        index +=1

plt.figure(figsize=(20,4))
for plotIndex, badIndex in enumerate(misclassifiedIndexes[0:5]):
    plt.subplot(1, 5, plotIndex + 1)
    plt.imshow(np.reshape(dev_data[badIndex], (28,28)), cmap=plt.cm.gray)
    plt.title('Predict: {}, Actual: {}'.format(predictions[badIndex], dev_labels[badIndex]), fontsize = 15)

Can you please take a look to see what is going wrong with my code? Thank you!

Answer 1

As such I cannot issue with your code. Hence, I'm providing here a reproducible code.

You can use the np.where on the Boolean comparison between predictions and actuals.

Try this example:

from sklearn.neighbors import KNeighborsClassifier
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

X, y = load_digits(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4)

NN_model = KNeighborsClassifier(n_neighbors=1)
NN_model.fit(X_train, y_train)
# Create the confusion matrix for the dev data
from sklearn.metrics import confusion_matrix
predictions = NN_model.predict(X_test)
confusion = confusion_matrix(y_test, predictions)

import matplotlib.pyplot as plt
misclassifiedIndexes = np.where(y_test!=predictions)[0]


fig, ax = plt.subplots(4, 3,figsize=(15,8))
ax = ax.ravel()
for i, badIndex in enumerate(misclassifiedIndexes):
    ax[i].imshow(np.reshape(X_test[badIndex], (8, 8)), cmap=plt.cm.gray)
    ax[i].set_title(f'Predict: {predictions[badIndex]}, '
                    f'Actual: {y_test[badIndex]}', fontsize = 10)
    ax[i].set(frame_on=False)
    ax[i].axis('off')
plt.box(False)
plt.axis('off')