在R中的随机森林树的每个最终节点处提取类分布

Question

I'm using randomForest package in R. From what I've understood,this package only gives me the assigned class to each instance at the final nodes of each tree, but I need to know the class distribution at each node.

Let's say at a final node of one tree we have 10 instances of class 0, and 20 instances of class 1 when the whole forest is trained. Now instead of saying the assigned class for this node is 1 (because of majority of instances from class 1) I want to know the class counts (10 and 20). is there any way to do so? Thanks for your help in advance.

Answer 1

You can use function predict.randomForest(.., type = "prob") to get the predicted probabilities. However, they are calculated by aggregating the predictions (but not the predicted probabilities!) of individual decision trees. If you have 10 trees predicting class=1 and 30 trees predicting class=0 , then this function call would yield the predicted probability of the first class as 0.25.

If you require "true" tree-level probabilities, then you must switch to a different RF algorithm. For example, Scikit-Learn's class RandomForestClassifier works this way.

Answer 2

Good question!

It would only be an issue for RF classification if you do not grow the trees fully. To prevent fully grown trees you would have to set minnodes>1 than 1 and/or maxnodes< N.samples. The randmForest implementation only stores the classification node prediction by its majority vote prediction, see getTree(rf)

I had the same issue writing the forestFloor package visualizing feature contributions . I had to recalculate all node states of the trees with a recursive Rcpp function. I think you have to do the same or fix the source code of the package. You could also ask Liaw, the maintainer of randomForest, to implement it. Or ask me to implement an output of the computed node states. A small chance one of the other random forest implementations already do support more detailed node states.

randomForest only output/stores majority vote of terminal notes

library(randomForest)
set.seed(123)
obs=2000
X = matrix(rnorm(obs))
y = factor((X+rnorm(obs))>=0)

plot(X,col=y)

rf = randomForest(X,y,
                  keep.inbag=T,
                  nodesize = 15,
                  ntree=2)

#but notice prob predictions only can be 100%, 50% or 0%
print(head(predict(rf,X,type="prob"),15)) #(NB these predictions are not OOB-CV!)

   FALSE TRUE
1    1.0  0.0
2    0.0  1.0
3    0.0  1.0
4    0.0  1.0
5    0.5  0.5
6    0.0  1.0
7    0.0  1.0
8    1.0  0.0
9    1.0  0.0
10   1.0  0.0
11   0.0  1.0
12   0.0  1.0
13   0.0  1.0
14   0.0  1.0
15   1.0  0.0