Graphics Reference
In-Depth Information
n
′
n
n
his leads to the eigenvalues-eigenvectors problem for
K.Denotingits
largest eigenvalue by α
(note that the multiplicative factor n is absorbed into the
eigenvalue) and its associated eigenvector by β
, then the corresponding first kernel
principal component is given by h
I
n
−
n
j
=
β
j
γ
j
in the feature space
κ
.Wecan
then sequentially findthe second, third,etc., principal components. From(
.
),we
have that β
k
, k
=
H
,
,...,areorthogonalto
n
and sothe normalization β
′
=
k
Kβ
k
=
is
n
′
n
n
equivalentto β
′
k
.hus, β
k
isnormalizedaccordingto α
k
β
′
I
n
−
Kβ
k
=
k
β
k
=
.
R
p
and its feature image γ
For an x
x
κ
, the projection of γ
x
along the
(
)H
(
)
kth eigencomponent of C
n
is given by
n
j
=
n
j
=
γ
(
x
)
, h
k
=
γ
(
x
)
,
β
kj
γ
j
=
β
kj
κ
(
x
j
, x
)
,
(
.
)
H
κ
H
κ
n
′
n
n
where β
k
is the kth eigenvector of
)
onto the dimension reduction linear subspace spanned by the leading r eigencom-
ponents of C
n
is given by
I
n
−
K. herefore, the projection of γ
(
x
n
j
=
n
j
=
)
R
r
.
β
j
κ
x
j
, x
,...,
β
rj
κ
x
j
, x
(
)
(
Let us demonstrate the idea behind KPCA using a few examples. here are three
data sets in this demonstration, the synthesized “two moon” data set, the “Pima dia-
betes” data set, and the “image segmentation” data set.
Example
5
First, we compare PCA and KPCA using the synthesized “two moons”
data set shown in Fig.
.
. he original data set is located in a
-D space in (a). We
can see that the two classes of data arenot well-separated along any one-dimensional
component. herefore, upon applying PCA, we are not going to see good separation
along the first principal coordinate axis. In the histogram (b
), the horizontal axis is
the first principal coordinate from the PCA and the vertical axis is the frequency. As
we can see, there is a great deal of overlap between two classes. A kernelized pro-
jection can provide a solution to this problem. In the histogram (b
), the horizontal
axis is the first principal coordinate given by KPCA (with the polynomial kernel of
degree
) and the vertical axis is the frequency. Note that the KPCA with polyno-
mial kernel also does not give good separation. However, in the histogram (b
), the
results obtained from KPCA using a radial basis function (RBF, also known as the
“Gaussian kernel”) with σ
5
are shown, and they exhibit a good separation. If ei-
ther PCA or KPCA is to be used as a preprocessing step before a classification task,
clearly KPCA using the radial basis function with σ
=
is the best choice. he ROC
curves for these approaches are shown in (c), with the area under the curve (AUC)
reported as
=
A
=
.
,
A
=
.
and
A
=
.
. KPCA with
PCA
KPCA
(
Poly
)
KPCA
(
RBF
)
RBF(σ
) is clearly better at distinguishing between two groups than classical PCA
and the KPCA using the polynomial kernel.
=
