Learning Mixtures of Independent Component Analysers - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 59

are of unknown class (unsupervised learning). Actually, we consider a more
general case in which knowledge of pC k = x ð n Þ

could be available for a subset of
k n pairs (supervised learning) and unknown for the rest of the k n pairs
(unsupervised learning).
We approach the problem of estimating A k ; b k and p ð s k Þ from a log-likelihood
perspective and assuming a non-parametric model for p s ðÞ: We demonstrated that
the derivates of all the unknown parameters W k ; b k (W is a compact notation) can
be expressed as [ 5 ],

pC ðÞ

¼ X

N

dlog j detW k j p W k x ð n Þ b k j detW k j p W k x ð n Þ b k dL x = W ð Þ

P

dW k dW k

K

n ¼ 1

j detW k 0 j p W k x ð n Þ b k 0 ð ð Þ Þ

k 0 ¼ 1

k ¼ 1...K

ð 3 : 5 Þ

Equating the derivative of the log-likelihood to zero leads to the set of equations that
are to be solved in order to find the parameters W k ; b k k ¼ 1...K : Unfortunately, a
closed form solution for the set of equations obtained in this way is unattainable. First
of all we need an expression for the source pdf's p s ðÞ k ¼ 1...K : In ICA, it is
assumed that p s ðÞ¼ ps k ðÞ ps k ð Þ ps k ð ; where s k ¼ s k1 s k2 ...s k ½ T and M is
the dimension of the data (number of sources). This simplifies the problem, but we
still need an expression for the marginal pdf's. Once the expression for the pdf's is
given, a highly nonlinear set of equations must be solved. This unavoidably leads to
iterative methods, as explained in the next section.

3.2 Iterative Solutions

Iterative solutions are based on decoupling the computation of the parameters from
the computation of p s ðÞ k ¼ 1...K (or of any other function deduced from
p s ðÞ k ¼ 1...K ; if needed). We consider the steps given in Table 3.1 .

3.3 A General Procedure for ICAMM

3.3.1 Non-Parametric Estimation of the Source pdf's

Non-parametric estimation of the source pdf's is the most general way of
approaching this problem since no particular parametric models are needed.
Imposing independence p s ðÞ
k

¼ ps ð n Þ
k1

ps ð n Þ
k2

...ps ð n Þ
kM

; we have to estimate

m ¼ 1...Mk ¼ 1...K : This can be done by means of [ 6 ]

the marginals ps ð n Þ
kM