CLASS IMBALANCE LEARNING METHODS FOR SUPPORT VECTOR MACHINES - Imbalanced Learning: Foundations, Algorithms, and Applications - page 87

5.5.1 Different Error Costs (DEC)

As pointed out in Section 5.3, the main reason for the SVM algorithm to be sen-
sitive to class imbalance would be that the soft margin objective function given
in Equation 5.10 assigns the same cost (i.e., C ) for both positive and negative
misclassifications in the penalty term. This would cause the separating hyperplane
to be skewed toward the minority class, which would finally yield a suboptimal
model. The different error costs (DEC) method is a cost-sensitive learning solu-
tion proposed in [8] to overcome this problem in SVMs. In this method, the SVM
soft margin objective function is modified to assign two misclassification costs,
such that C + is the misclassification cost for positive class examples, while C −
is the misclassification cost for negative class examples, as given in Equation
5.11. Here we also assume positive class to be the minority class and negative
class to be the majority class.

⎛
⎝

⎞
⎠

l l

1
2 w

C + C −

min · w + ξ i + ξ i

i | y i =+ 1 i | y i =− 1

s . t . i (w · (x i ) + b) ≥ 1 − ξ i (5.11)

ξ i ≥ 0 ,i = 1 ,...,l

By assigning a higher misclassification cost for the minority class examples
than the majority class examples (i.e., C + >C − ), the effect of class imbal-
ance could be reduced. That is, the modified SVM algorithm would not tend to
skew the separating hyperplane toward the minority class examples to reduce the
total misclassifications, as the minority class examples are now assigned with a
higher misclassification cost. The dual Lagrangian form of this modified objective
function can be represented as follows:

⎧
⎨

⎫
⎬

l l l

1
2

max
α i

α i − α i α j y i y j K(x i ,x j )

⎩ ⎭

i = 1 i = 1 j = 1

l

0 ≤ α i ≤ C + , 0 ≤ α i ≤ C − ,i = 1 ,...,l s . t . y i α i = 0 , (5.12)

i = 1

where α i and α i represent the Lagrangian multipliers of positive and negative
examples, respectively. This dual optimization problem can be solved in the
same way as solving the normal SVM optimization problem. As a rule of thumb,
Akbani et al. [10] have reported that reasonably good classification results could
be obtained from the DEC method by setting the C − /C + equal to the minority-
to-majority-class ratio.