Decision Forests - Data Mining with Decision Trees: Theory and Applications - page 147

instead of selecting features one should select the ensemble's members [ Liu
et al . (2004) ] .
One can also rank the classifiers according to their ROC performance.
Then, they suggest to plot a graph where the Y -axis displays a performance
measure of the integrated classification. The X -axispresentsthenumberof
classifiers that participated in the combination. i.e. the first best classifiers
from the list are combined by voting (assuming equal weights for now) with
the rest getting zero weights. The ensemble size is chosen when there are
several sequential points with no improvement.
The GASEN algorithm was developed for selecting the most appro-
priate classifiers in a given ensemble [ Zhou et al . (2002) ] . In the initial-
ization phase, GASEN assigns a random weight to each of the classifiers.
Consequently, it uses genetic algorithms to evolve those weights so that
they can characterize to some extent the fitness of the classifiers in joining
the ensemble. Finally, it removes from the ensemble those classifiers whose
weight is less than a predefined threshold value.
Recently, a revised version of the GASEN algorithm called GASEN-b
has been suggested [Zhou and Tang (2003)]. In this algorithm, instead of
assigning a weight to each classifier, a bit is assigned to each classifier
indicating whether it will be used in the final ensemble. In an experimental
study, the researchers showed that ensembles generated by a selective
ensemble algorithm, which selects some of the trained C4.5 decision trees
to make up an ensemble, may be not only smaller in size but also
stronger in the generalization than ensembles generated by non-selective
algorithms.
A comparative study of pre-combining and post-combining methods
when meta-combining methods are used has been performed in [ Prodro-
midis et al . (1999) ] . The results indicate that the post-combining pruning
methods tend to perform better in this case.
9.7 Cross-Inducer
This property indicates the relation between the ensemble technique and
the inducer used.
Some implementations are considered as an inducer-dependent type,
namely these ensemble generators which use intrinsic inducer, have been
developed specifically for a certain inducer. They can neither work nor
guarantee effectiveness in any other induction method. For instance, the
works of [ Hansen (1990); Lu and Ito (1999); Sharkey (1996) ] were developed