决策树分类器如何与全局约束一起使用？

Question

I generated a decision tree classifier with sklearn in Python, which works well in terms of accuracy. I train the classifier with the optimal solution of a linear program, which returns an optimal assignment of items to classes while, considering a global cost-constraint (ie assigning item 1 to class A comes at a cost of x. Total resulting costs over all items and classes need to be smaller than a value y).

After reclassifying all items with the classifier, while the accuracy is acceptable, the global cost-constraint is violated in most classification runs. Naturally so, since the standard decision tree from sklearn in python does not consider the constraint.

Is there a way to incorporate global constraints to be upheld after classification ? Is there a way to force the tree to consider all already classified items when making the next assignment choice? I assume this would require to establish some sort of cost- or penalty-function to be checked during classification by the tree.

Answer 1

Decision trees as implemented in sklearn are built only based on a splitting criteria that considers Gini coefficient, entropy or information gain. Custom loss functions are not possible.

However Gradient Boosted Trees, such as XGboost, LightGBM and CatBoost allow to specify your own loss functions. A tutorial can be found here: https://towardsdatascience.com/custom-loss-functions-for-gradient-boosting-f79c1b40466d

You would then incorporate a penalty term for violating your constraint into the loss function.