Decision Forests - Data Mining with Decision Trees: Theory and Applications - page 99

Chapter 9

Decision Forests

9.1 Introduction

The main idea of an ensemble methodology is to combine a set of models,
each of which solves the same original task, in order to obtain a better
composite global model, with more accurate and reliable estimates or
decisions than can be obtained from using a single model. The idea of
building a predictive model by integrating multiple models has been under
investigation for a long time. In fact, ensemble methodology imitates our
second nature to seek several opinions before making any crucial decision.
We weigh individual opinions, and combine them to reach our final decision
[ Polikar (2006) ] .
It is well known that ensemble methods can be used for improving
prediction performance. Researchers from various disciplines such as statis-
tics, machine learning, pattern recognition, and data mining considered the
use of ensemble methodology. This chapter presents an updated survey of
ensemble methods in classification tasks. The chapter provides a profound
descriptions of the various combining methods, ensemble diversity generator
and ensemble size determination.
In the literature, the term “ensemble methods” is usually reserved for
collections of models that are minor variants of the same basic model.
Nevertheless, in this chapter we also cover hybridization of models that
are not from the same family. The latter is also referred in the literature as
“multiple classifier systems.”

9.2 Back to the Roots

Marie Jean Antoine Nicolas de Caritat, marquis de Condorcet (1743-1794)
was a French mathematician who among others wrote in 1785 the Essay on

99