sklearn roc_auc_score multiclass ovo average error

Question

The sklearn 1.1.2 doc says for function roc_auc_score

average=None is only implemented for multi_class='ovo'

However, when I try to run

from sklearn.metrics import roc_auc_score

y_true = [0, 1, 2, 3]
y_pred = [0, 1, 2, 3]
y_pred_proba = [
    [0.7, 0.1, 0.1, 0.1],
    [0.1, 0.7, 0.1, 0.1],
    [0.1, 0.1, 0.7, 0.1],
    [0.1, 0.1, 0.1, 0.7],
]

roc_auc_score(y_true, y_pred_proba, multi_class="ovo", average=None)

I got the following Error:

NotImplementedError: average=None is not implemented for multi_class='ovo'

This is a bug? Can't find any mention of this issue anywhere.

I am using sklearn verison 1.1.2:

>>> sklearn.__version__
'1.1.2'

Answer 1

Probably just typo, not a bug. It can be seen in code for _multiclass_roc_auc_score (which is called in roc_auc_score function in case of multiple classes) and from meaning of average parameter. So average == None should return whatever was computed, and for ovo case it's matrix, which kind of doesn't match with purpose of this roc-auc function: return either one value (as in case with average=macro ) or return value for each class ( multi_class=ovr and average=None ).

By the way, ovo -case matrix still may be gained by adopting _average_multiclass_ovo_score function with one change -- insted of return np.average(pair_scores) should be return pair_scores .