Boosting Algorithm of Ensemble Learning in Machine Learning

Boosting Algorithm of Ensemble Learning:

The most widely used ensemble method is called Boosting. It works on the concept of a weighted training set. In such a training set, each example has an associated weight, w_i≥0.

Working of Boosting Algorithm:

Working Boosting starts with w_j=1 for all examples, training set. From this, the first hypothesis, h_f is generated. The hypothesis will classify some of the training examples correctly and some incorrectly. For the next hypothesis to perform better on the miss classified example, the weights of the miss classified ones are increased. From this new weighted training set, the hypothesis, h₂ is generated.

The process is continued until the M hypothesis has been generated where M is the input of the boosting algorithm. The final ensemble hypothesis is a weighted-majority combination of the M hypothesis, each weighted according to how it will perform on the training set.