Graphics Reference
In-Depth Information
KPCA.herefore,wehavealsoprovidedfiguresretainingonly“brickface”and“path.”
hese clearly show that separation is produced by KPCA but not by the PCA.
Remark
1
he selection of the kernel and its window width is still an issue in gen-
eral kernel methodology. here are some works that provide guidance on kernel se-
lection for classification and supervised learning problems, but there is still a lack of
guidelinesinclusteringandunsupervisedlearning problems.Inthiswork,wemerely
aimed to show that nonlinear information about data can be obtained by applying
kernel methods with only minor effort required. he kernel method can help to dig
out nonlinear information about the data, which would be di
cult or impossible to
obtain by applying classical linear PCA in the original input space.
1
Kernel Canonical Correlation Analysis
10.4
Researchershavelongbeeninterestedindescribingandclassifyingrelations between
two sets of variables. Hotelling (
) introduced canonical correlation analysis to
describe the linear relation between two sets of variables that have a joint distribu-
tion. his defines a new coordinate system for each of the sets such that the new pair
of coordinate systems are the best at maximizing correlations. he new systems of
coordinates are simply linear systems of the original ones. hus, classical CCA can
only be used to describe linear relations. Using such linear relations, classical CCA
canonly findlineardimension reduction subspaceandlinear discriminant subspace.
However,motivated by the active development and the popular and successful usage
of various kernel machines, a hybrid approach combining classical CCA with a ker-
nel machine (Akaho,
; Bach and Jordan,
), named kernel canonical correla-
tion analysis, has emerged in recent years. KCCA has also been studied recently by
Hardoon et al. (
) among others.
Suppose a random vector X with p components has a probability distribution P
R
p
. We partition X into
on
X⊂
X
()
X
=
,
()
X
with p
and p
components, respectively. he corresponding partition of
X
is de-
.Weare interested in finding relations between X
()
and X
()
.Clas-
sical CCA is concerned with linear relations. It describes linear relations byreducing
the correlation structure between these two sets of variables to the simplest possible
form by means of linear transformations of X
()
and X
()
. It finds pairs
notedby
X
X
(
α
i
, β
i
)
R
p
+
p
in the following way. he first pair maximizes the correlation between α
′
X
()
