Learning Mixtures of Independent Component Analysers - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 79

1 1 1

0.5 0.5 0.5

0 0 0

-0.5 -0.5 -0.5

-1 -1 -1

-1 0 1 -1 0 1 -1 0 1

1 1 1

(a) (b) (c)

sr=0 sr=0.3 sr=1

Fig. 3.10 Decision regions in ICA mixtures for different supervision ratios (sr)

effect of increasing outliers is masked, i.e., some observation vectors, which are
supposed to be outliers, in fact belong to the generative data model. Including
them in training would improve the resulting estimated model.
Finally, Fig. 3.10 shows two class decision regions in one of the generated ICA
mixtures that were estimated with different supervision ratios. Figure 3.10 a, b
represent decision regions for unsupervised training and a low-supervision ratio
training, respectively. Figure 3.10 c shows the decision regions for supervised
training. The accuracy of the decision regions improved with higher supervision
since only observation vectors belonging to the generative data model were used in
the training stage.

3.5 Conclusions

A novel procedure so-called Mixca for learning the parameters of mixtures of
independent component analyzers (mixture matrices, centroids, and source prob-
ability densities) has been introduced. The proposed method provides a versatile
framework with the following characteristics: no assumptions about the densities of
the original sources are required; mixtures with nonlinear dependencies and semi-
supervised learning are considered; and any ICA algorithm can be incorporated for
updating the model parameters. Considering this last characteristic, a set of variants
depending on the embedded algorithm used for ICAMM parameter updating can be
defined, e.g., Mixca, Mixca-JADE, Mixca-FastIca, etc. The suitability of applica-
tion of the proposed technique has been demonstrated in several ICA mixtures and
ICA datasets. The non-parametric approach of the procedure clearly yielded better
results in source separation than standard ICA algorithms, indicating promising
adaptive properties in learning source densities using small sample sizes. In