Learning Mixtures of Independent Component Analysers - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 57

The method based on Beta divergence proposed by Mollah et al. [ 2 ]is
developed from a different approach than the one proposed. Mollah's method is a
sequential method that extracts one ICA parameter set at a time while the proposed
method is a block extraction technique that estimates the parameters for each ICA.
The differences between the proposed method and Lee's method are also appli-
cable for Mollah's method. In addition, Mollah's method requires application of
various parameters (beta value, classification percentage for stopping criterion,
etc.), which are estimated rather arbitrarily, making the method unstable. There-
fore, it follows that the convergence to the optimum is not guaranteed.
The method proposed by Choudrey and Roberts [ 3 ] is based on variational
Bayesian inference. This method defines priors over all possible parameters (ICA
mixture indicator variables, ICA mixture coefficients, mixture proportions, mean
and precision over each MoG, bias vector, etc.) of the ICA mixture. This kind of
heavy parameterization requires great knowledge about the data. Broad priors have
little or no effect on the results, but they can reduce their regularizing abilities
leading to implausible magnitudes and possible over-fitting. Narrow priors encode
strong assumptions about possible parameter and variable values; the model
becomes inflexible and its ability to learn is compromised [ 3 ]. Thus, finding the
right priors requires a great examination consisting in searching for an extensive
number of parameters in a wide range of values. Another relevant feature of
Choudrey and Roberts's method is that the source model for every ICA is MoG,
and thus the final ICAMM data model is a kind of MoG. Choudrey and Roberts's
method search for generalization, but tuning of the extensive number of prior
parameters seems to be at best complicated. In contrast, the proposed method
attempts to find a balance between parametric and non-parametric estimation.
Thus, the ICA mixtures are modelled by a short set of parameters maintaining
simplicity, but the source density estimation is flexible since it is non-parametric.
In addition, the non-gaussianity of the data is preserved since any assumption
about the source model is not imposed. This modelling is especially plausible in
semi-supervised scenarios where fragmented knowledge is available as is common
in real-world problems. With regard to the automatic estimation of the number of
clusters, we can deal with this issue using the hierarchical algorithm proposed in
Chap. 4 . By defining a high number of clusters (ICA mixtures) at the bottom level
of the hierarchy, an optimum number for clustering can be estimated at interme-
diate hierarchy levels using some stopping rules or cluster validation techniques.
From the above discussion, it is clear that are significant differences in the
comparison of the previous methods with the proposed one. A competitive com-
parison of the proposed method with the previous methods by processing different
cases is always desirable. However, since the implementation or adaptation of the
previous methods would require great effort and the preparation of testing data sets
was not straightforward considering the different perspectives from which the
previous methods were made, we decided to study in greater depth our ICAMM
method and apply it in different areas, instead of trying to compare it with other
methods. This decision also provided the opportunity to apply our method to novel
applications involving real-world problems.