Scikit学习随机森林分类器：如何根据树的数量生成OOB错误图

Question

---

Edit 2 : There is now a lovely example in the sklearn documentation on this.

---

In order to see how many trees are necessary in my forest, I'd like to plot the OOB error as the number of trees used in the forest is increased. I'm in Python using a sklearn.ensemble.RandomForestClassifier but I can't find how to predict using a subset of trees in the forest. I could do this by making a new random forest on each iteration with increasing numbers of trees but this is too expensive.

It seems a similar task is possible with the Gradient Boosting object using the staged_decision_function method. See this example .

This is quite a simple procedure in R and can be achieved by simply calling plot(randomForestObject) :

---

-- Edit -- I see now the RandomForestClassifier object has an attribute estimators_ which returns all the DecisionTreeClassifier objects in a list. So to solve this I can iterate through that list, predicting the results from each tree and taking a 'cumulative average'. However, is there really no easier way to do this already implemented?

Answer 1

There is a discussion and code in this issue: https://github.com/scikit-learn/scikit-learn/issues/4273

You can add trees one-by-one like this:

n_estimators = 100
forest = RandomForestClassifier(warm_start=True, oob_score=True)

for i in range(1, n_estimators + 1):
    forest.set_params(n_estimators=i)
    forest.fit(X, y)
    print i, forest.oob_score_

The solution you propose also needs to get the oob indices for each tree, because you don't want to compute the score on all the training data.

I still feel this is a strange thing to do as the is really no natural ordering of the trees in the forest. Can you explain what you use-case is? Do you want to find the minimum number of trees for a given accuracy to reduce prediction time? If you want fast prediction time, I'd suggest using GradientBoostingClassifier, which is usually much faster.