Plot maximum-margin hyperplane in 3-space with Python

Question

I have found this really cool notebook on SVM :

https://github.com/jakevdp/sklearn_tutorial/blob/master/notebooks/03.1-Classification-SVMs.ipynb

But was a bit frustrated 'cause it doesn't show how to plot the maximum-margin in 3-space. I've tried my best but have no clue about how to do it... For instance I'd like to plot it on the following reprsentation (which is taken from the notebook):

Also, if suport vectors could be encircled it would be the sherry on the top.

Answer 1

This is a very good question, however the notebook provided is actually "lying". This not how kernel works, and while this visualization is cool, it is not what happens "inside SVM". Lets put this aside though and focus on how to plot 3D separation hyperplane in this projected space (which is not RBF projected space).

All you have to do is:

1. Fit linear SVM to the 3D data used for this plot.
2. Extract weights (clf.coefs_) and bias (clf.intercept_)
3. Plot 3d hyperplane with normal (clf.coefs_) and distance from the origin (clf.intercept_)

Answer 2

import numpy as np
from sklearn.svm import SVC
from sklearn.datasets import make_circles

X_1_2, y = make_circles(100, factor = .1, noise=.1)
X_3 = np.exp(-(X_1_2[:,0] ** 2 + X_1_2[:,1] ** 2))
X = np.insert(X_1_2, 2, X_3, axis=1)

clf = SVC(kernel='linear').fit(X,y)

w = clf.coef_
w1 = w [:, 0]
w2 = w [:, 1]
w3 = w [:, 2]
b = clf.intercept_
sv = clf.support_vectors_

import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

xx, yy = np.meshgrid(range(-1,2), range(-1,2))
zz = (-w1 * xx - w2 * yy - b) * 1. /w3

%matplotlib notebook
plt3d = plt.subplot(projection='3d')
plt3d.plot_wireframe(xx, yy, zz, rstride=1, cstride=1, color='purple')
plt3d.scatter3D(X[:, 0], X[:, 1], X[:, 2], c=y, s=50, cmap='winter')
plt3d.scatter3D(sv[:, 0], sv[:, 1], sv[:, 2], s=150)
plt.show()