Decision Forests - Data Mining with Decision Trees: Theory and Applications - page 100

the Application of Analysis to the Probability of Majority Decisions. This
work presented the well-known Condorcet's jury theorem. The theorem
refers to a jury of voters who need to make a decision regarding a binary
outcome (for example, to convict or not a defendant). If each voter has a
probability p of being correct and the probability of a majority of voters
being correct is M then:

• p> 0 . 5 implies M>p .

• Also M approaches 1, for all p> 0 . 5 as the number of voters approaches

infinity.

This theorem has two major limitations: the assumption that the
votes are independent; and that there are only two possible outcomes.
Nevertheless, if these two preconditions are met, then a correct decision
can be obtained by simply combining the votes of a large enough jury
that is composed of voters whose judgments are slightly better than a
random vote.
Originally, the Condorcet Jury Theorem was written to provide a
theoretical basis for democracy. Nonetheless, the same principle can be
applied in pattern recognition. A strong learner is an inducer that is given
a training set consisting of labeled data and produces a classifier which can
be arbitrarily accurate. A weak learner produces a classifier which is only
slightly more accurate than random classification. The formal definitions
of weak and strong learners are beyond the scope of this topic. The reader
is referred to [ Schapire (1990) ] for these definitions under the PAC theory.
A decision stump inducer is one example of a weak learner. A Decision
Stump is a one-level Decision Tree with either a categorical or a numerical
class label.
For the sake of clarity, let us demonstrate the ensemble idea by applying
it to the Labor dataset presented in Table 9.1. Each instance in the table
stands for a collective agreement reached in the business and personal
services sectors (such as teachers and nurses) in Canada during the years
1987-1988. The aim of the learning task is to distinguish between acceptable
and unacceptable agreements (as classified by experts in the field). The
selected input-features that characterize the agreement are:

• Dur — the duration of agreement

• Wage — wage increase in first year of contract

• Stat — number of statutory holidays

• Vac — number of paid vacation days