Information Technology Reference
In-Depth Information
Clearly, if the variables x
1
and x
2
are independent, then the F - correlation is equal
to zero. It can be shown that F - correlation is the maximal possible correlation
between one-dimensional linear projections U
ð
x
1
Þ
and U
ð
x
2
Þ
, with U
ð
x
Þ¼
K
ð;
x
Þ
being the feature map, where the kernel K
ð;
x
Þ
is a function in F for each x.Thisis
the definition of the first ''canonical correlation'' between U
ð
x
1
Þ
andU
ð
x
2
Þ
.
Canonical correlation analysis (CCA) is a multivariate statistical technique
similar to PCA. While PCA works with a single random vector and maximizes the
variance of projections of the data, CCA works with a pair of random vectors (or in
general with a set of m random vectors) and maximizes correlation between sets of
projections. While PCA leads to an eigenvector problem, CCA leads to a gen-
eralized eigenvector problem.
Kernel-ICA employs a ''kernelized'' version of CCA to compute a flexible
contrast
function
for
ICA.
The
following
definitions
are
considered.
Let
and
denote
x
1
;
...
;
x
1
x
2
;
...
;
x
2
sets
of
N
observations
of
x
1
and
x
2
,
denote the
corresponding images in feature space. Let S
1
and S
2
represent the linear spaces
spanned by the a
i
-images of the data points. Thus, f
1
¼
P
U x
1
;
...
;
U x
1
and U x
2
;
...
;
U x
2
respectively, and let
a
1
U x
1
þ
f
1
N
and
k
¼
1
f
2
¼
P
N
a
2
U
ð
x
2
Þþ
f
2
, where f
1
and f
2
are orthogonal to S
1
and S
2
, respectively.
k
¼
1
Considering that K
1
and K
2
are the Gram matrices associated with the data sets
x
i
1
and
x
i
2
, respectively, the following variance estimates are obtained:
^
^
U x
ð ;
f
h ð Þ ¼
N
a
2
K
2
K
2
a
2
. Thus, the
kernelized CCA problem for two variables becomes that of performing the fol-
lowing maximization:
Þ ¼
N
a
1
K
1
K
1
a
1
var
ð
h
U x
ð ;
f
1
i
and
var
a
1
K
1
K
2
a
2
a
1
K
1
a
1
q
F
K
1
;
K
2
ð
Þ¼
max
a
1
;
a
2
2
R
ð
2
:
33
Þ
1
=
2
1
=
2
N
a
2
K
2
a
2
The formulation as a generalized and regularized eigenvalue problem to m
variables is the following:
0
@
1
A
2
0
@
1
A
K
1
þ
N
2
I
K
1
K
2
...
K
1
K
m
a
1
a
2
.
a
m
2
K
2
þ
N
2
K
2
K
1
I
...
K
2
K
m
.
.
.
2
K
m
þ
N
2
K
m
K
1
K
2
K
m
...
I
0
@
1
A
ð
2
:
34
Þ
2
0
@
1
A
K
1
þ
N
2
I
0
...
0
a
1
a
2
.
a
m
2
K
1
þ
N
2
0
I
...
0
¼
k
.
.
.
2
K
1
þ
N
2
0
0
...
I
Search WWH ::
Custom Search