ASSESSMENT METRICS FOR IMBALANCED LEARNING - Imbalanced Learning: Foundations, Algorithms, and Applications - page 196

Classifier that
is always wrong

1

Positive
trivial
classifier

Negative
trivial
classifier

Error
rate

Classifier A1

Classifier B2 Classifiers

B at different
thesholds

Classifiers
A at different
thresholds

0.25

Classifier that
is always right

1

0 0.4

P ( + ), Probability of an example
being from the positive class

Figure 8.3 The cost curves illustration for two hypothetical scoring classifiers A 1 and
B 2 , in which a single classifier is not strictly dominant throughout the operating range.

a line, and the convex hull of the ROC space corresponds to the lower envelope
created by all the classifier lines.
In cost space, the relative expected misclassification cost is plotted as a func-
tion of the probability of an example belonging to the positive class. Since the
point in the ROC space where two curves cross is also represented in cost space
by the crossing of two lines, this point can easily be found in cost space and its
x -coordinate read off the graph. This value corresponds to the imbalance ratio at
which a switch from the classifier that dominated the ROC space up to the curve
crossing (say, classifier A) to the classifier that started dominating the ROC space
afterward (say, classifier B) is warranted. The cost curves thus make it very easy
to obtain this information. In contrast, ROC graphs tell us that, sometimes, A is
preferable to B, but one cannot read off when this is so, from the ROC graph.

8.4.3 Precision-Recall Curves
PR curves are similar to ROC curves in that they explore the trade-off between
the well-classified positive examples and the number of misclassified negative
examples. As the name suggests, PR curves plot the precision of the classifier
as a function of its recall, as shown in Figure 8.4. In other words, it measures
the amount of precision that can be obtained as various degrees of recall are
considered. For instance, in the domain of document retrieval systems, PR curves
would plot the percentage of relevant documents identified as relevant against
the percentage of relevant documents deemed as such with respect to all the