ICA and ICAMM Methods - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 42

N 1
m 1

N 1 X

m N þ 1

m

H m spacings Z 1 ; ... ; Z N

Z ð m ð i þ 1 Þþ 1 Þ Z ð mi þ 1 Þ ð 2 : 30 Þ

log

i ¼ 0

m

Under the condition that m ; N !1;

N ! 0 ; this estimator is consistent; typically

m ¼ N

p

. The intuition behind this estimator is that by considering m spacings
with larger and larger values of m, the variance of the probability mass of these
spacings relative to their expected values gets smaller and smaller. In fact, the
probability mass of m spacings is distributed according to a beta distribution
with parameters m and N þ 1[ 50 ]. Thus, a modification of Eq. ( 2.30 ) in which the
m spacings overlap is used in Radical. The final contrast consists of an entropy
estimator that is used to minimize Eq. ( 2.8 ),

X

N m

1
N m

N þ 1
m

H Radical Z 1 ; ... ; Z N

Z ð i þ m Þ Z ð i Þ log ð 2 : 31 Þ

i ¼ 1

The optimization method of the algorithm for cost function minimization is
exhaustive search. It is assumed that the data are first pre-whitened and augmented
with a number of synthetic replicates of each of the original N sample points with
additive spherical Gaussian noise to make a surrogate data set. This is done in
order to obtain a smoother version of the estimator in an attempt to remove false
minima. Afterwards, for each angle h, the data are rotated s ¼ W ð h Þð Þ using a
pair-wise Jacobi rotation and the cost function evaluated. The output is the W
corresponding to the optimal h. There are M ð M 1 Þ= 2 distinct Jacobi rotations
parameterized by h (for a M-dimensional ICA). Optimizing over a set of these
rotations is known as a sweep. Empirically, performing multiple sweeps improves
the estimate of W for some number of iterations. In [ 50 ], good results were
reported in simulations for S M(S is the number of sweeps).

2.3.3 Kernel-ICA

The Kernel-ICA algorithm [ 48 ] uses contrast functions based on canonical cor-
relations in a reproducing kernel Hilbert space. This approach is not based on a
single nonlinear function, but rather on an entire function space of candidate
nonlinearities. The contrast function is a rather direct measure of the dependence
of a set of random variables. Considering the case of two univariate random
variables x 1 and x 2 , and letting F be a vector space of functions from , the

F-correlation qF is defined as the maximal correlation between the random
variables f 1 ð x 1 Þ and f 1 ð x 1 Þ , where f 1 and f 2 range over F:

R to R

cov f 1 x ð ; f 2 x ðÞ ð Þ

q F ¼ max
f 1 ; f 2 2 F

corr f 1 x ð ; f 2 x ðÞ ð Þ ¼ max

f 1 ; f 2 2 F

ð 2 : 32 Þ

Þ 1 = 2 Þ 1 = 2 ð varf 1 x ðÞ ð varf 2 x ðÞ