error with sklearn CalibratedClassifierCV and SVM

Question

I want to use sklearn's CalibratedClassifierCV in conjuction with sklearn's SVC to make predictions for a multiclass (9 classes) prediction problem. However when I run it, I get the following error. This same code will run no problem with a different model (ie RandomForestCalssifier).

kf = StratifiedShuffleSplit(y, n_iter=1, test_size=0.2)
clf = svm.SVC(C=1,probability=True)            
sig_clf = CalibratedClassifierCV(clf, method="isotonic", cv=kf)
sig_clf.fit(X, y)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/g/anaconda/lib/python2.7/site-packages/sklearn/calibration.py", line 166, in fit
    calibrated_classifier.fit(X[test], y[test])
  File "/home/g/anaconda/lib/python2.7/site-packages/sklearn/calibration.py", line 309, in fit
    calibrator.fit(this_df, Y[:, k], sample_weight)
IndexError: index 9 is out of bounds for axis 1 with size 9

Answer 1

This is a problem of the SVC using a One-vs-One strategy, and therefore the decision function having shape (n_samples, n_classes * (n_classes - 1) / 2) . A possible workaround would be do to CallibratedClassifierCV(OneVsRestClassifier(SVC())) . If you want to use sigmoidal calibration, you can also do SVC(probability=True) and not use CallibratedClassifierCV .

We should fix the SVC decision function, I think.