Computational Intelligence Techniques - Computational Intelligence in Biomedical Engineering - page 97

Lagrangian function is a convex function if the objective function ( w )is

convex and the space defined by the constraint set of G ( w ) and H ( w ) is con-
vex. In optimization, convexity is often synonymous with unique and global
solutions, which is very desirable because we can easily test for them and then
stop the algorithms.
With that in mind, we note the inequality constraint in Equation 3.34 and
write the Lagrangian primal problem as follows:

m n n

= 1
2

w i + C

L p ( w , α , ξ )
min

ξ i − π i ξ i

i =1 i =1 i =1

⎛ ⎛ ⎞ ⎞

n m

⎝ y i ⎝ ⎠ − ⎠

− α i w j φ j ( x i )+ b 1+ ξ i (3.36)

i =1 j =1

where ∀ i =1 ,...,n

α i ,π i ≥ 0

The primal form of Equation 3.36 offers a dual representation, which is another
way of representing the equation in terms of Lagrange multipliers alone. The
dual representation is found by differentiating Equation 3.36 with respect to
the variables and finding the stationary conditions. The partial gradients with
respect to the variables are

n

∂ L p
∂ w i

= w i − α j y j φ i ( x j )

j =1

⎛ ⎞

m

∂ L p
∂ α i

⎝ ⎠ −

= y w j φ j ( x i )+ b 1+ ξ i

j =1

n

∂ L p
∂ b

= α i y i

i =1

∂ L p
∂ ξ i

= C − α i − π i

Using the stationary conditions of the partial gradients, that is, setting them to
zero, we then eliminate the variables by substitution to obtain the Lagrangian
dual only in terms of Lagrangian multipliers α i

n n

α i + 1
2

L D ( α )
min

= − α i α j y i y j K ( x i , x j )

i =1 i,j =1