Learning Mixtures of Independent Component Analysers - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 62

ns

x ð ns Þ pC ks = x ð ns Þ ; W

b ks ð 0 Þ¼ ns ks 2½ 1 ; K ns 2½ 1 ; N ð 3 : 9 Þ

pC ks = x ð ns Þ ; W ð Þ

where ks ns extends only to the pairs k n where there is prior knowledge about
pC k = x ð n Þ ; W

: For those categories to be learned in a totally unsupervised form,
simply initialize the centroids randomly. Actually, Eq. ( 3.9 ) could be used in any
iteration for all k n pairs as an alternative to Eq. ( 3.8b ), as suggested in [ 1 ].

3.3.3 Using Any ICA Algorithm

The computation of the gradient in Eq. ( 3.8a )or( 3.8c ) clearly has two separate
factors. The first one is related to the underlying ICA model of every class, while
the second one is related to ICAMM. The first factor is

x ð n Þ T

1 þ fs ðÞ
k

W k

or

W k in Eq. ( 3.8a )or( 3.8c ) respectively. The second factor
may be thought of as a weighting factor that defines the relative degree of cor-
rection of the parameters in a particular iteration: corrections are proportional to
the (estimated or known) pC k = x ð n Þ ; W

s ð n Þ

k T

I þ fs ð n Þ
k

for every class. Recognizing this fact
leads naturally to the conclusion that any of the many ICA alternative algorithms
could be used.
The general ICAMM algorithm (including non-parametric source pdf estima-
tion, supervised-unsupervised learning, and possibility of selecting a particular
ICA algorithm) is in Table 3.2 . The correction of residual dependence in the
classification stage of the algorithm is explained in the next section.

3.3.4 Correction of the Conditioned Class-Probability
After Convergence

The learning stage of the algorithm in Table 3.2 stops when iterations do not
significantly change the updated parameter estimates. Let us assume that this is
done at iteration i ¼ I. Then, if a new feature vector is to be classified, we need to
compute p ð C k = x Þ using Eq. ( 3.4 ), with the final parameter and source estimates, i.e.,

p ð C k = x Þ¼ j det W k ð I Þj p ð s k Þ p ð C k Þ
P

s k ¼ W k ð I Þð x b k ð I ÞÞ k ¼ 1 ; ... ; K

K

j det W k 0 ð I Þj p ð s k 0 Þ p ð C k Þ

k 0 ¼ 1

ð 3 : 10 Þ