How is decision boundary for ensemble methods/decision tree generated in machine learning?

Question

Decision Trees work by splitting the training subsets at every node iterating through root to leaf nodes till we get our classification/regression result.

But how is the actual decision boundary(weights) calculated? We use the cost function to perform the split at every node. Does this cost function also help us to find weights?

For example, to build an AdaBoost classifier, a first base classifier (such as a Decision Tree) is trained and used to make predictions on the training set. The relative weight of misclassified training instances is then increased. A second classifier is trained using the updated weights and again it makes predictions on the training set, weights are updated, and so on.

How is this Relative weight calculated?

Answer 1

In Adaboosting the relative weights are calculated as follows,

First, given m number of training instances, every instance is given an equal weight of 1/m.

Now we define the weighted error rate of the jth predictor/classifier after training as follows:

r(j) = sum(incorrect instance's weights) / sum(all instance's weights)

Now we define another term, the predictor/classifier's weight as follows :

cw(j) = learning rate * log ( (1 - r(j))/(rj) )

Now to the relative weights of each instance, we calculate them as follows, where i is the instance's index/identifier:

if w(i) was correctly classified in the last predictor, then :

w(i) = w(i)

else if w(i) was incorrectly classified, then :

w(i) = w(i) * exp(cw(j))

This would increase the weights of misclassified instances if they are less than 50% of the instances, if the last predictor classified 50% of the instance's correct, this would set r(j) = .5, which leads to cw(j) = 0, and w(i) for the incorrect instances would be w(i) * 1, which means no changes in the incorrect instances weights.