Learning Mixtures of Independent Component Analysers - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 61

dL ð X = W Þ
dW k

¼ dL ð X = W Þ
dW k

W k W k

p ð C k Þ

W k

j det W k j p s ð n Þ

s ð n Þ
k

T

¼ X

N

k

I þ fs ð n Þ
k

ð 3 : 8c Þ

P

K

j det W k 0 j p s ð n Þ

n ¼ 1

k 0

k 0 ¼ 1

Then we can apply Eqs. ( 3.8a , b , c ) in the gradient updating algorithm of Eq. (3.6)
to iteratively find the parameters W k ; b k ; k ¼ 1...K :
This algorithm is connected to the one proposed in [ 1 ], but here the nonlinear function
f ð s k Þ is the one corresponding to a non-parametric estimation of the source pdf. Note
that both f ð s ðÞ
k

Þ and p ð s ðÞ
k

Þ are actually computed in a non-parametric manner

2
4

! 2 3

5

s ð n Þ

km s ð n 0 Þ

1
2

km

n 0 6¼ n s ð n 0 Þ

h

e

¼ 1
h 2

km

fs ðÞ
km

s ð n Þ
km

(kernel density estimates) using

! 2

km s ð n 0 Þ

s ð n Þ

1
2

km

n 0 6¼ n e

h

and ( 3.7 ) respectively, given a suitable algorithm to a more extensive field of appli-
cations. The estimation is asymptotically unbiased and efficient, and it is shown to
converge to the true pdf under several measures, when a suitable kernel is chosen [ 6 ].
The parameter h ; which controls the smoothness of the functional f ; was estimated as
h ¼ 1 : 06rN 1 = 5 r ¼ std s ðÞ m ¼ 1... ð Þ; which is the normal reference rule
using a Gaussian kernel [ 6 ]. We use this value of h in simulations of Sect. 3.5 .

3.3.2 Unsupervised-Supervised Learning

We consider a hybrid supervised-unsupervised situation in which we could know
a priori pC k = x ð n Þ ; W

for some k n pairs. This is the most general form to define
the possible prior knowledge about the training set in a probabilistic context. For
example,

pC k 0 = x ð n Þ ; W

¼ 1 and

x ð n Þ

if is of known class C k 0 ; then

¼ 0 k 6¼ k 0 ; but we can think of more general cases. For those
k n pairs where there is prior knowledge about pC k = x ð n Þ ; W

pC k = x ð n Þ ; W

; we can avoid the

k ðÞ

j det W k j p s ð n Þ

computation of in Eqs. ( 3.8a , b , c ) for all the iterations, using the

P

K

j det W k 0 j p s ð n Þ

k ðÞ

instead.
To help in the correct initialization of the categories in this general supervised-
unsupervised method, it is convenient to select the initial centroids in the form

k 0 ¼ 1

known pC k = x ð n Þ ; W