plot calibration curve for machine learning

Question

I have the code below and this code work only with the binary class so how can I use with three classes.

from sklearn.tree import DecisionTreeClassifier
import pandas as pd
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import scikitplot as skp


orgnal_data = pd.read_excel("movie.xls")

# Program extracting first column
text = orgnal_data.iloc[:,0]
lable = orgnal_data.iloc[:,1]

x_train,x_test,y_train,y_test=train_test_split(fe,lable,test_size=0.30,random_state=40)


DT = DecisionTreeClassifier()
DT_y = DT.fit(x_train,y_train).predict(x_test)

clf_names = ['Decision Tree']
skp.metrics.plot_calibration_curve(y_test,DT_y,clf_names)
plt.show()

Answer 1

Since you use scikit-plot module, there is no function for a multiclass problem.

Read the source code here :

This function currently only works for binary classification.

So you can either 1) modify the source code or 2) open a github issue and request a function for multiclass problems.

EDIT 1:

Using scikit-learn you have some ML models that can handle multiclass problems. For example for the LinearSVC function here , the multiclass support is handled according to a one-vs-the-rest scheme.

So you can actually have models like this and then use the plot_calibration_curve function for each case (one VS rest) separately.