ENSEMBLE METHODS FOR CLASS IMBALANCE LEARNING - Imbalanced Learning: Foundations, Algorithms, and Applications - page 64

class label must be with more than half of the votes, and it always outputs the
class label with the most of the votes. Weighted voting considers that different
learners perform differently. It empowers stronger learners to influence more in
voting.
Different from averaging and voting that use pre-specified combination rules,
stacking trains a learner to combine base learners as a general learning procedure.
Base learners are called the first-level learners , and the combiner is called the
second-level learner or meta-learner . Briefly speaking, base learners are trained
from the original training data, while the second-level learner is trained with
different training data, where an example takes the base learners' output as input
(attribute values), and the original class label as output.
4.3 ENSEMBLE METHODS FOR CLASS IMBALANCE LEARNING
In CIL, the ground-truth level of imbalance is often unknown, and the ground-
truth relative importance of the minority class against the majority class is often
unknown either. There are many potential variations; so it is not strange that
ensemble methods have been popularly adopted to achieve more effective and
robust performance, including Bagging, boosting, Random Forests (RF) [19],
stacking, and so on. According to what ensemble method is involved, ensem-
ble methods for CIL can be roughly categorized into Bagging-style methods,
boosting-based methods, hybrid ensemble methods, and the others.
Existing CIL methods can be improved by adopting ensemble learning. Under-
sampling can solve class imbalance problems effectively and efficiently. How-
ever, it is wasteful to ignore potential useful information contained in the majority
class examples, as it is often expensive to gather training examples. Great benefits
will be obtained if the ignored examples are exploited appropriately. On the other
hand, ensemble methods can benefit from the diversity of data. So, it is natural
to use ensemble methods to further explore the examples ignored by under-
sampling. Several ensemble methods are devoted to improve under-sampling,
including Chan and Stolfo's method [5], EasyEnsemble and BalanceCascade [6],
UnderBagging [20], and many others.
Over-sampling replicates the majority class examples; so it has the risk of
over-fitting. Some ensemble methods are designed to help reduce the risk. For
example, DataBoost-IM [8] uses boosting architecture to identify hard examples
in each round and creates synthetic examples for the minority class according
to the level of imbalance of hard examples. Synthetic minority over-sampling
technique (SMOTE) [21] is a state-of-the-art CIL method that reduces the risk of
over-fitting by introducing synthetic examples instead of making replicates, but
it could introduce noise. To improve SMOTE, SMOTEBoost [4] is proposed. In
each boosting round, the weight distribution is adjusted by SMOTE to focus more
on the minority class examples. Thus, the weak learners that could be affected
by noise can be boosted into strong learners via ensemble.
In the following, we introduce some typical ensemble methods for CIL.