confusion matrix for multilabel classification using prefetch dataset

Question

I am new to keras and machine learning in general and I have adapted this keras tutorial to my case that involves text classification into 8 categories.

In this tutorial a prefetch dataset is created for training and testing, but it does not show how to create a confusion matrix from prefetch datasets. I followed the instructions on this https://www.pythonfixing.com/2021/11/fixed-how-to-plot-confusion-matrix-for.html which also involves multi label classification (iris dataset with 3 classes) but I am getting an error:

ValueError                                Traceback (most recent call last)
<ipython-input-81-02116c95e29c> in <module>()
----> 1 confusion_matrix(predicted_categories, true_categories)

1 frames
/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_classification.py in 
_check_targets(y_true, y_pred)
     93         raise ValueError(
     94             "Classification metrics can't handle a mix of {0} and {1} 
targets".format(
---> 95                 type_true, type_pred
     96             )
     97         )

ValueError: Classification metrics can't handle a mix of multiclass and multilabel- 
indicator targets

My code is below:

epochs = 20

shallow_mlp_model = make_model()
shallow_mlp_model.compile(
    loss="binary_crossentropy", optimizer="adam", metrics=["categorical_accuracy"]
)

history = shallow_mlp_model.fit(
    train_dataset, validation_data=validation_dataset, epochs=epochs
)


def plot_result(item):
    plt.plot(history.history[item], label=item)
    plt.plot(history.history["val_" + item], label="val_" + item)
    plt.xlabel("Epochs")
    plt.ylabel(item)
    plt.title("Train and Validation {} Over Epochs".format(item), fontsize=14)
    plt.legend()
    plt.grid()
    plt.show()


plot_result("loss")
plot_result("categorical_accuracy")

_, categorical_acc = shallow_mlp_model.evaluate(test_dataset)
print(f"Categorical accuracy on the test set: {round(categorical_acc * 100, 2)}%.")


predictions=shallow_mlp_model.predict(test_dataset)

predicted_categories = tf.argmax(predictions, axis=1)

true_categories = tf.concat([y for x, y in test_dataset], axis=0)

confusion_matrix(predicted_categories, true_categories)

So, anyone could help me to create a confusion matrix in this case?

Answer 1

I have found the solution. My true labels were still one-hot-encoded. So, I used tf.argmax in the variable "true_categories" and the confusion matrix was printed.

true_categories=tf.argmax(true_categories, axis=1)
confusion_matrix(predicted_categories, true_categories)