scikit-learn 中 Adaboost 中使用的决策函数的定义（公式）是什么

Question

I find that sklearn.ensemble.AdaBoostClassifier there is a method decision_function .

I think this function should return a value related to the possibility of a sample belongs to a category. Am I right?

But I can't find the formula for decision function in AdaBoostClassifier , does anyone know that?

Answer 1

It is located exactly where it is supposed to be - in the source of AdaBoostClassifier, currently in the line 639

https://github.com/scikit-learn/scikit-learn/blob/51a765a/sklearn/ensemble/weight_boosting.py#L639

def decision_function(self, X):
    """Compute the decision function of ``X``.
    Parameters
    ----------
    X : {array-like, sparse matrix} of shape = [n_samples, n_features]
        The training input samples. Sparse matrix can be CSC, CSR, COO,
        DOK, or LIL. DOK and LIL are converted to CSR.
    Returns
    -------
    score : array, shape = [n_samples, k]
        The decision function of the input samples. The order of
        outputs is the same of that of the `classes_` attribute.
        Binary classification is a special cases with ``k == 1``,
        otherwise ``k==n_classes``. For binary classification,
        values closer to -1 or 1 mean more like the first or second
        class in ``classes_``, respectively.
    """
    check_is_fitted(self, "n_classes_")
    X = self._validate_X_predict(X)

    n_classes = self.n_classes_
    classes = self.classes_[:, np.newaxis]
    pred = None

    if self.algorithm == 'SAMME.R':
        # The weights are all 1. for SAMME.R
        pred = sum(_samme_proba(estimator, n_classes, X)
                   for estimator in self.estimators_)
    else:   # self.algorithm == "SAMME"
        pred = sum((estimator.predict(X) == classes).T * w
                   for estimator, w in zip(self.estimators_,
                                           self.estimator_weights_))

    pred /= self.estimator_weights_.sum()
    if n_classes == 2:
        pred[:, 0] *= -1
        return pred.sum(axis=1)
    return pred