ASSESSMENT METRICS FOR IMBALANCED LEARNING - Imbalanced Learning: Foundations, Algorithms, and Applications - page 192

for the F -measure, although it could, theoretically, avoid this problem by manip-
ulating its parameter, it is not clear what value that parameter should take when
the misclassification costs are not exactly known. To overcome these problems,
Batuwita and Palade [10] proposed the AGm. Its definition is as follows:

G − mean + specificity × N n
1 + N n

AGm = , ifspecificity > 0 (8.11)

AGm = 0 , ifsensitivity = 0 (8.12)

This metric is more sensitive to variations in specificity than in sensitivity. Fur-
thermore, this focus on specificity is related to the number of negative examples
in the dataset so that the higher the degree of imbalance, the higher the reaction
to changes in specificity. The authors showed the advantages of this new measure
as compared to the G-mean and the F -measure.

8.3.6.4 Index of Balanced Accuracy The designers of the index of balanced
accuracy (IBA) [11] respond to the same type of problem of the G-mean (as well
as of the AUC) described by Batuwita and Palade [10]. In particular, they show
that different combinations of sensitivity and specificity can lead to the same
values for the G-mean. Their generalized IBA is defined as follows:

IBA α (M) = ( 1 + α × Dom ) × M (8.13)

where dominance (Dom) is defined as:

sensitivity − specificity (8.14)

where M is any metric, and α is a weighting factor designed to reduce the
influence of the dominance on the result of a particular metric M . IBA α was
tested both experimentally on artificial and University of California, Irvine (UCI)
data and theoretically. It is shown to have low correlation with accuracy (which is
not appropriate for class-imbalanced datasets), but high correlation with G-mean
and AUC (which are appropriate for class-imbalanced datasets). It is, therefore,
indeed, geared at the right type of problems.

8.4 RANKING METHODS AND METRICS: TAKING UNCERTAINTY
INTO CONSIDERATION

An important disadvantage of all the threshold metrics discussed in the previous
section is that they assume full knowledge of the conditions under which the
classifier will be deployed. In particular, they assume that the class imbalance
present in the training set is the one that will be encountered throughout the oper-
ating life of the classifier. If that is truly the case, then the previously discussed
metrics are appropriate; however, it has been suggested that information related