Decision boundaries for nearest centroid

Question

I am trying to draw decision boundaries for different classifiers including the nearestcentroid , but when I use this code

if hasattr(clf, "decision_function"):
            Z = clf.decision_function(np.c_[xx.ravel(), yy.ravel()])
        else:
            Z = clf.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:, 1]

I get an error saying 'NearestCentroid' object has no attribute 'predict_proba' . How can I fix this?

Answer 1

Assuming your X has two features, you can generate a meshgrid where each axis pertains to one of the features.

Assuming X is your features array with two features - shape would be (N, 2), where N is the number of samples - and y is your target array.:

# first determine the min and max boundaries for generating the meshgrid
feat1_min, feat1_max = X[:, 0].min() - 1, X[:, 0].max() + 1
feat2_min, feat2_max = X[:, 1].min() - 1, X[:, 1].max() + 1

Now generate your meshgrid and make predictions along the grid:

xx, yy = np.meshgrid(np.arange(feat1_min, feat1_max , 0.02),
                     np.arange(feat2_min, feat2_max , 0.02))  # 0.02 is step size
Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])

Now make the plot:

Z = Z.reshape(xx.shape)
plt.figure()
plt.pcolormesh(xx, yy, Z, cmap="autumn")
plt.scatter(X[:, 0], X[:, 1], c=y, cmap="autumn",
            edgecolor='k', s=10)
plt.show()

Answer 2

As BearBrown pointed out, you only check if "decison_function" is an attribute of clf . You never check if "predict_proba" is an attribute of clf

if hasattr(clf, "decision_function"):
    Z = clf.decision_function(np.c_[xx.ravel(), yy.ravel()])
elif hasattr(clf, "predict_proba"): # This condition ensures that you'll see that predict_proba is not an attribute of clf`enter code here`
    Z = clf.predict_proba(numpy.c_[xx.ravel(), yy.ravel()])[:, 1]
else: #This will show you your error again
    raise AttributeError("Neither 'decision_function' not 'predict_proba' found in clf")

After this, you should check why what you expect is not an attrib of clf

Answer 3

You can make your own predict_proba :

from sklearn.utils.extmath import softmax
from sklearn.metrics.pairwise import pairwise_distances

def predict_proba(self, X):
    distances = pairwise_distances(X, self.centroids_, metric=self.metric)
    probs = softmax(distances)
    return probs

clf = NearestCentroid()
clf.predict_proba = predict_proba.__get__(clf)
clf.fit(X_train, y_train)
clf.predict_proba(X_test)