Decision Forests - Data Mining with Decision Trees: Theory and Applications - page 116

• For selection rule 3, if the classifications of the two lower levels are not

equal, the classification made by the sub-arbiter based on the first group
is chosen. In case this is not true and the classification of the sub-arbiter
constructed on the third group equals those of the lower levels, then
this is the chosen classification. In any other case, the classification of
the sub-arbiter constructed on the second group is chosen. In fact it is
possible to achieve the same accuracy level as in the single mode applied
to the entire dataset but with less time and memory requirements [ Chan
and Stolfo (1993) ] . More specifically, it has been shown that this meta-
learning strategy required only around 30% of the memory used by the
single model case. This last fact, combined with the independent nature
of the various learning processes, make this method robust and effective
for massive amounts of data. Nevertheless, the accuracy level depends on
several factors such as the distribution of the data among the subsets and
the pairing scheme of learned classifiers and arbiters in each level. The
decision regarding any of these issues may influence performance, but the
optimal decisions are not necessarily known in advance, nor initially set
by the algorithm.

9.3.2.3 Combiner Trees
The way combiner trees are generated is very similar to arbiter trees.
Both are trained bottom-up. However, a combiner, instead of an arbiter,
is placed in each non-leaf node of a combiner tree [Chan and Stolfo
(1997) ] . In the combiner strategy, the classifications of the learned base
classifiers form the basis of the meta-learner's training set. A composition
rule determines the content of training examples from which a combiner
(meta-classifier) will be generated. In classifying an instance, the base
classifiers first generate their classifications and based on the composition
rule, a new instance is generated. The aim of this strategy is to combine the
classifications from the base classifiers by learning the relationship between
these classifications and the correct classification. Figure 9.6 illustrates the
result obtained from two base classifiers and a single combiner.
Two schemes for composition rules were proposed. The first one is
the stacking scheme. The second is like stacking with the addition of the
instance input attributes. It has been shown that the stacking scheme per se
does not perform as well as the second scheme [ Chan and Stolfo (1995) ] .
Although there is information loss due to data partitioning, combiner trees
can sustain the accuracy level achieved by a single classifier. In a few cases,
the single classifier's accuracy was consistently exceeded.