Regularized Kernel Locality Preserving Discriminant Analysis for Face Recognition - Advanced Concepts for Intelligent Vision Systems - page 286

Unfortunately, we don't know the explicit form of x i and u i in kernel space,
it is therefore impossible to compute W φ and B φ directly. Hence, the so-called
distance kernel trick is employed to solve this problem, which makes the distances
of vectors in kernel space be a function of the distance of input vectors, i.e.,

x i − x j x i − x j ,x i − x j

2 =

x i ,x i − x i ,x j x j ,x j = 2 +

= k ( x i ,x i ) − 2 k ( x i ,x j )+ k ( x j ,x j ) (7)

Define kernel matrices, K XX = k ( x i ,x j )] i,j =1 , K UU =[ k ( u i ,u j )] i,j =1 ,and
K XU =[ k ( x i ,u j )] i =1 j =1 ,expression (5) and (6) can be rewritten as

⎧
⎨
⎩

) , x i ∈

NN k ( x j )or x j ∈ NN k ( x i ),

K XX

ii − 2 K XX ij

+ K XX
jj

exp( −

w φk
ij

=

2 t 2

x i ,x j ∈ ω k .

0 ,

otherwise.

(8)

bij φ = exp(

K UU

ii − 2 K UU ij

+ K UU
jj

) , u i ∈ NN k ( u j )or u j ∈ NN k ( u i ). −

2 t 2

0 ,

otherwise.

(9)

Then, KLPDA is to maximize the following function

i,j =1 ( m i −

m j ) b ij ( m i − m j ) T

A T U φ H φ ( U φ ) T A
A T X φ L φ ( X φ ) T A

J ( A )= = (10)

k =1 y i ,y j ∈ ω k
y i = y j

y j ) w φk
ij

( y i − ( y i − y j ) T

where E φ and D φ are diagonal matrices with the diagonal entries being the
column or row ( B φ and W φ are symmetric) sums of B φ and W φ , H φ = E φ B φ

−

and L φ = D φ W φ are Laplacian matrices.

Since any solution of (10), a i ∈

−

F , must lie in the span of all the samples in

j =1 , such that a i = j =1 ψ ij x j

n

= X φ ψ i ,

F , there exist coecients ψ i = { ψ ij }

that is A = X φ Ψ . Thus, problem (10) can be converted to

J ( A )= Ψ T ( X φ ) T U φ H φ ( U φ ) T X φ Ψ
Ψ T ( X φ ) T X φ L φ ( X φ ) T X φ Ψ

= Ψ T K XU H φ ( K XU ) T Ψ
Ψ T K XX L φ ( K XX ) T Ψ

(11)

For convenience, we call S b = K XU H φ ( K XU ) T , S w = K XX L φ ( K XX ) T ,and
S t = S b + S w the kernel locality preserving between-class,within-class, and total
scatter matrix respectively. So the problem of (11) is converted to solve the
following generalized eigenvalue problem

S b Ψ = λS w Ψ (12)

The solution of (12) is consist by the d leading eigenvectors of ( S w ) − 1 S b .