FOUNDATIONS OF IMBALANCED LEARNING - Imbalanced Learning: Foundations, Algorithms, and Applications - page 33

did not impact the performance of the large disjuncts by using a different learn-
ing method classify them [11]. A similar hybrid approach was also used in one
additional study [39, 40].
Others advocate the use of instance-based learning for domains with many
rare cases/small disjuncts because of the highly specific bias associated with this
learning method [10]. In such methods, all training examples are generally stored
in memory and utilized, as compared to other approaches where examples, when
they fall below some utility threshold, are ignored (e.g., because of pruning). In
summary, there have been several attempts to select an inductive bias that will
perform better in the presence of small disjuncts that are assumed to represent
rare cases. But these methods have shown only mixed success and, most signif-
icantly, this work has not directly examined the class imbalance; these methods
may assist with the class imbalance as rare classes are believed to be formed
disproportionately from rare cases. Such approaches, which have not garnered
much attention in the past decade, are quite relevant and should be reexamined
in the more modern context of class imbalance.
2.4.3.4 Algorithms that Implicitly or Explicitly Favor Rare Classes and Cases
Some algorithms preferentially favor the rare classes or cases and hence tend
to perform well on classifying rare classes and cases. Cost-sensitive learning
algorithms are one of the most popular such algorithms for handling imbal-
anced data. While the assignment of costs in response to the problem charac-
teristics is done at the problem level, cost-sensitive learning must ultimately
be implemented at the algorithm level. There are several algorithmic meth-
ods for implementing cost-sensitive learning, including weighting the training
examples in a cost-proportionate manner [42] and building the cost sensitivity
directly into the learning algorithm [43]. These iterative algorithms place different
weights on the training distribution after each iteration and increase (decrease) the
weights associated with the incorrectly (correctly) classified examples. Because
rare classes/cases are more error-prone than common classes/cases [4, 38], it is
reasonable to believe that boosting will improve their classification performance.
Note that because boosting effectively alters the distribution of the training
data, one could consider it a type of advanced adaptive sampling technique.
AdaBoost's weight-update rule has also been made cost sensitive, so that mis-
classified examples belonging to rare classes are assigned higher weights than
those belonging to common classes. The resulting system, Adacost [44], has
been empirically shown to produce lower cumulative misclassification costs than
AdaBoost and thus, like other cost-sensitive learning methods, can be used to
address the problem with rare classes.
Boosting algorithms have also been developed to directly address the problem
with rare classes. RareBoost [45] scales false-positive examples in proportion
to how well they are distinguished from true-positive examples and scales
false-positive examples in proportion to how well they are distinguished from
true-negative examples. A second algorithm that uses boosting to address the
problems with rare classes is SMOTEBoost [46]. This algorithm recognizes that