CLASS IMBALANCE AND ACTIVE LEARNING - Imbalanced Learning: Foundations, Algorithms, and Applications - page 103

certain regions of the problem space; while a model may be able to accurately
identify large regions of the minority space, portions of this space may get misla-
beled, or labeled with poor quality, because of underrepresentation in the training
set. At the extreme, disjunctive subregions may get missed entirely. Both of these
problems are particularly acute as the class imbalance increases, and are discussed
in greater detail in Section 6.6. Finally, in the initial stages of AL, when the base
model is somewhat naıve, the minority class may get missed entirely as an AL
heuristic probes the problem space for elusive but critical minority examples.

6.2.3 Addressing the Class Imbalance Problem with Active Learning
As we will demonstrate in Section 6.3, AL presents itself as an effective strategy
for dealing with moderate class imbalance even without any special consid-
erations for the skewed class distribution. However, the previously discussed
difficulties imposed by more substantial class imbalance on the selective abilities
of AL heuristics have led to the development of several techniques that have
been specially adapted to imbalanced problem settings. These skew-specialized
AL techniques incorporate an innate preference for the minority class, leading
to more balanced training sets and better predictive performance in imbalanced
settings. Additionally, there exists a category of density-sensitive AL techniques,
techniques that explicitly incorporate the geometry of the problem space. By
incorporating the knowledge of independent dimensions of the unlabeled example
pool, there exists a potential for better exploration, resulting in improved resolu-
tion of rare subregions of the minority class. We detail these two broad classes
of AL techniques as follows.

6.2.3.1 Density-Sensitive Active Learning Utility-based selection strategies for
AL attribute some score, U ( · ) , to each instance x encapsulating how much
improvement can be expected from training on that instance. Typically, the
examples offering a maximum

U (x) are selected for labeling. However, the focus
on individual examples may expose the selection heuristic to outliers, individual
examples that achieve a high utility score, but do not represent a sizable por-
tion of the problem space. Density-sensitive AL heuristics seek to alleviate this
problem by leveraging the entire unlabeled pool of examples available to the
active learner. By explicitly incorporating the geometry of the input space when
attributing some selection score to a given example, outliers, noisy examples,
and sparse areas of the problem space may be avoided. The following are some
exemplary AL heuristics that leverage density sensitivity.

Information Density. This is a general density-sensitive paradigm compatible
with arbitrary utility-based active selection strategies and a variety of metrics
used to compute similarity [12]. In this case, a meta-utility score is computed
for each example based not only on a traditional utility score, U (x) , but also
on a measurement of that example's similarity to other instances in the problem