Graphics Reference
In-Depth Information
Let
Z=
Φ
(X)
,withistermedthefeature space. he inner product in
Z
is given by
Φ
(
x
Φ
(
u
)=
q
λ q ϕ q
(
x
)
ϕ q
(
u
)=
κ
(
x,u
)
( . )
he kernel trick ( . ) of turning inner products in
into kernel values allows us
to carry out many linear methods in the spectrum-based feature space
Z
without
needing to know the spectrum Φ itself explicitly. herefore, it makes it possible to
constructnonlinear(fromtheEuclideanspaceviewpoint)variantsoflinearmethods.
Consider another transformation,
Z
γ
XH
κ given by γ
(
x
)=
κ
(
x,
ċ)
,
( . )
which brings a point in
X
to an element in
H
κ .heoriginalsamplespace
X
is thus
embedded into anew sample space
κ . he map is called an Aronszajn map in Hein
and Bousquet ( ). We connect these two maps ( . ) and ( . ) via
H
J
Φ
(X)
γ
(X)
,givenby
J(
Φ
(
x
)) =
κ
(
x,
ċ)
.Notethat
J
is a one-to-one linear transforma-
tion satisfying
Z
H κ
H κ .
Φ
(
x
)
=
κ
(
x, x
)=
κ
(
x,
ċ)
=
γ
(
x
)
hus, Φ
areisometrically isomorphic,andthese two feature represen-
tations ( . )and ( . )are equivalent in this sense. Since they are equivalent, math-
ematically there is no distinction between them. However, from a data visualization
perspective, there is a difference. As the feature map ( . ) is not explicitly known,
there is no way of visualizing the feature data in
(X)
and γ
(X)
. In this article, for data visualiza-
tion purposes, data or extracted data features are placed in the frameworkof
Z
κ .We
will use the feature map ( . ) for the later KPCA and kernel canonical correlation
analysis (KCCA).Sincethedataclusterwillbevisualized intheoriginal sample space
X
H
for the SVC, we will use the spectrum-based feature map ( . ) for ease.
Given data
x ,...,x n
, let us write, for short, the corresponding new data in the
feature space
H
κ as
γ
(
x j
)=
γ j
( H
)
.
( . )
κ
As can be seen later, via these new data representations ( . ) and ( . ), statisti-
cal procedures can be solved in this kernel feature space
κ in a parallel way us-
ing existing algorithms of classical procedures such as PCA and CCA. his is the
key idea behind kernelization. he kernelization approach can be regarded, from
the original sample space viewpoint, as a nonparametric method, since it adopts
a model via kernel mixtures. It still has the computational advantage of keeping the
process analogous to a parametric method, as its implementation only involves solv-
ing a parametric-like problem in
H
κ . he resulting kernel algorithms can be inter-
preted as running the original parametric (oten linear) algorithms on kernel feature
space
H
κ . For the KPCA and KCCA in this article, we use existing PCA and CCA
codes on kernel data. One may choose to use codes from MATLAB,R, Splus or SAS.
he extra programming effort involved is the preparation of data in an appropriate
kernel form.
H
Search WWH ::




Custom Search