Decision Forests - Data Mining with Decision Trees: Theory and Applications - page 140

constructed. Feature set partitioning, on the other hand, decomposes the
original set of features into several subsets and builds a classifier for each
subset. Thus, a set of classifiers is trained such that each classifier employs
a different subset of the original feature set. Subsequently, an unlabeled
instance is classified by combining the classifications of all classifiers.
Several researchers have shown that the partitioning methodology can
be appropriate for classification tasks with a large number of features
[ Kusiak (2000) ] . The search space of a feature subset-based ensemble
contains the search space of feature set partitioning, and the latter contains
the search space of feature selection. Mutually exclusive partitioning has
some important and helpful properties:
(1) There is a greater possibility of achieving reduced execution time
compared to non-exclusive approaches. Since most learning algorithms
have computational complexity that is greater than linear in the
number of features or tuples, partitioning the problem dimensionality
in a mutually exclusive manner means a decrease in computational
complexity [ Provost and Kolluri (1997) ] .
(2) Since mutual exclusiveness entails using smaller datasets, the classifiers
obtained for each sub-problem are smaller in size. Without the mutually
exclusive restriction, each classifier can be as complicated as the classi-
fier obtained for the original problem. Smaller classifiers contribute to
comprehensibility and ease in maintaining the solution.
(3) According to Bay (1999), mutually exclusive partitioning may help
avoid some error correlation problems that characterize feature sub-
set based ensembles. However, Sharkey (1996) argues that mutually
exclusive training sets do not necessarily result in low error correlation.
This point is true when each sub-problem is representative.
(4) In feature subset-based ensembles, different classifiers might generate
contradictive classifications using the same features. This inconsistency
in the way a certain feature can affect the final classification may
increase mistrust among end-users. We claim that end-users can grasp
mutually exclusive partitioning much easier.
(5) The mutually exclusive approach encourages smaller datasets which
are generally more practicable. Some data mining tools can process
only limited dataset sizes (for instance, when the program requires that
the entire dataset will be stored in the main memory). The mutually
exclusive approach can ensure that data mining tools can be scaled
fairly easily to large datasets [ Chan and Stolfo (1997) ] .