Learning Mixtures of Independent Component Analysers - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 63

Table 3.2 Proposed ICAMM algorithm—the Mixca procedure
Initialization
0. Initialize i ¼ 0 ; W k ð 0 Þ; b k ð 0 Þ: Use Eq. ( 3.9 ) to initialize b k ð 0 Þ for those classes that have
supervision in the training set. Select an ICA algorithm
Learning stage
1.

k ¼ 1...Kn ¼ 1...N

Compute s ð n k ð i Þ¼ W k ð i Þ x ðÞ b k ð i Þ

ð i Þ¼ pC k = x ð n Þ ; W for those k n pairs with knowledge about

2.

Directly use pC k = x ð n Þ ; W

p ð C k Þ

j det W k ð i Þj p s ð n k ð i Þ

. Compute pC k = x ð n Þ ; W ð i Þ¼

pC k = x ð n Þ ; W k ¼ 1...K

P

K

j det W k 0 ð i Þj p s ð n Þ

k 0 ð i Þ

k 0 ¼ 1

for the rest of the k n pairs. Use Eq. ( 3.7 ) to estimate p s ð n k ð i Þ

3.

Use the selected ICA algorithm to compute the increments D ð n Þ

ICA W k ð i Þ corresponding to the
observation x ð n Þ ; n ¼ 1...N ; which would be applied in W k ð i Þ in an ''isolated'' learning of
class C k . Compute the total increment by DW k ð i Þ¼ P

ð i Þ .

N

D ð n Þ

ICA W k ð i Þ pC k = x ð n Þ ; W

n ¼ 1

Update W k ð i þ 1 Þ¼ W k ð i Þþ a DW k ð i Þ k ¼ 1...K.

h h i w km ð i Þ pC k = x ð n Þ ; W i

4.

Compute Db k ð i Þ using Db k ð i Þ¼ P

ð i Þ

N

diag fs ð n Þ
k

n ¼ 1

2
4

! 2 3

5

km s ð n 0 Þ

s ð n Þ

1
2

km

n 0 6¼ n s ð n 0 Þ h

e

¼ 1
h 2

km

Use fs ð n Þ
km

s ð n Þ
km

to estimate fs ð n Þ
k

! 2

km s ð n 0 Þ

s ð n Þ

1
2

km

n 0 6¼ n e h

Actualize b k ð i þ 1 Þ¼ b k ð i Þþ b Db k ð i Þ k ¼ 1...k ; or re-estimate

ð i Þ

P

N

x ð n Þ pC k = x ð n Þ ; W

n ¼ 1

b k ð i þ 1 Þ¼ k ¼ 1...K

P

N

pC k = x ð n Þ ; W ð Þð i Þ

n ¼ 1

5. Go back to step 1, with the new values W k ð i þ 1 Þ; b k ð i þ 1 Þ and i ! i þ 1
Classification stage
6. Assuming learning stage stops at iteration i ! I. For a new feature vector to be classified,

estimate p ð C k = x Þ¼ j det W k ð I Þj p ð s k Þ p ð C k Þ
P

s k ¼ W k ð I Þð x b k ð I ÞÞ k ¼ 1 ; ... ; K

K

j det W k 0 ð I Þj p ð s k 0 Þ p ð C k Þ

k 0 ¼ 1

7. Estimate the source pdf using

! 2

s km s ð n Þ

1
2

km ð I Þ

p ð s k Þ¼ p ð s k1 Þ p ð s k2 Þ p ð s kM Þ where p ð s km Þ¼ a P

N

h

e or using

n ¼ 1

multidimensional density estimation (considering residual dependence)

0
@

1
A

h i T h i

s k s ð n Þ
k

s k s ð n Þ
k

ð I Þ ð I Þ

1
2 h 0

p ð s k Þ¼ a 0 P

N

e

n ¼ 1