Digital Signal Processing Reference
In-Depth Information
4
A Kernel-Induced Space Selection Approach to Model
Selection
As we know, class separation information should be preserved as much as possible in
the optimal kernel feature space
F
. Due to
F
changing with kernel function para-
(
)
(
)
(
)
(
)
(
)
()
meter and
is trace of matrix),
LPKHDA model selection criterion established is given as following
tr
AB
=
tr
BA
,
tr
AB
+=
tr
A
+
tr
B
(
tr
⋅
(
)
(
)
(
)
(
)
()
()
(
)
(
)
CS
θ
=
tr
S
φ
L
−
1
−
λ
tr
S
φ
L
1
−
λ
tr
S
φ
+
η
N
(15)
Ξ
w
w
C
(
)
(
)
where,
S X S X
are total and with-
in-class scatter matrices with locality preserving, they are formulated as
(
S
φ
L
=
12
N
X S
φ
X
φ
T
and
φ
Ξ
=
L
12
N
φ
φ
T
w
l
l
() ()
l
l
l
xx
l
=
1
xx
(
)
(
)
(
)
)
N
φ
Ξ
L
(
)
()
()
tr
S
=
φ
L
l
T
l
tr
S
=
l
1
N tr
D
K
−
1
K
1
,
.
w
l
() ()
N
S
N
l
l
l
=
1
xx
l
l
(
)
(
)
2
(
)
(
)
(
)
()
()
()
()
T
N
l
l
l
l
1
Nr
DK
−
1K1
.where
K
=
k
x
,
x
S
,
xx
S
N
S
i
j
ij
ij
,
=
, ,
,
N
l
(
)
(
)()
2
()
()
()
x
l
l
l
S
Kx
=
k
,
x
S
,
K
=
k
x
,
x
. In practical applica-
i
j
ij
i
j
ij
,
=
, ,
,
N
ij
,
=
, ,
,
N
l
tion, RBF function is often employed as kernel function. In order to ensure the numer-
ical stability of
(
)
tr
S
L
. The kernel function is modified as Ref[15] . The criterion in
w
()
(15) turns out to
CS
μ
θ
. The optimal
θ
*
is obtained through gradient descent. De-
'
()
rivate
CS
μ
θ
with respect to
θ
, the gradient is
'
(
)
(
)
(
)
()
∂
tr
S
φ
L
∂
tr
S
φ
L
∂
tr
S
φ
L
∂
CS
θ
(
)
(
)
(
)
()
(
)
(
)
()
2
Ξ
w
w
μ
'
φ
φ
L
φ
L
φ
=
−
β
β
tr
S
+
η
N
−
β
tr
S
−
β
tr
S
β
tr
S
+
η
N
w
Ξ
w
w
∂
θ
∂
θ
∂
θ
∂
θ
(
)
(
)
(
)
φ
L
T
N
∂
tr
S
∂
θ
=
1
N tr
D
∂
K
∂
θ
−
1
∂
K
∂
θ
1
Ξ
xx
S
N
(
)
(
)
(
)
N
()
()
l
l
∂
tr
S
φ
L
∂
θ
=
1
1
N tr
D
∂
tr
K
∂
θ
−
1
T
∂
tr
K
∂
θ
1
l
w
l
() ()
N
S
N
l
l
l
=
xx
l
l
(
)
N
2
2
()
2
3
where
∂
tr
K
∂
θ
=
x
−
x
exp
−
x
−
x
2
θ
θ
,
∂∂=
S
K
σ
i
j
i
j
ij
,
=
1
N
(
)
(
)
()
(
2
)
()
()
x
l
l
Sk
∂
xx
,
∂
θ
.
∂
tr
K
∂
θ
and
∂
tr
K
∂
θ
can be deduced in a
ij
i
j
S
ij
,
=
1
similar way.
In this paper, we must search for optimal
(
)
**
,
in parameter space. Inspired
from the idea in Ref [6], we propose an alternative LPKHDA method called
boosted LPKHDA by introducing Adaboost idea into LPKHDA. The implementation
procedure of Boosted LPKHDA is similar to Ref [3].
λη
