Introduction - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 19

source vectors after convergence. This accounts for possible non-linearities that
are not included in the basic ICA linear model. Modelling of the residual
dependencies will allow a correction of the probabilities of every class given the
feature vector, thus improving the classification performance.
The third feature included in Mixca is semi-supervised learning when both
labelled and unlabelled data are available to learn the model parameters. Semi-
supervised learning has been extensively studied for different classification
frameworks and classical generative models such as transductive support vector
machines, graph-based methods, hidden Markov random fields and Bayesian
classifiers [ 72 ], and in the context of the information theoretic learning [ 73 , 74 ].
Recently, applications of BSS and ICA have been adapted to incorporate any
available prior knowledge about the source signals (locations, statistics, and so on)
into an approach called semi-blind source separation (SBSS). This is done by
imposing, for instance, temporal or spatial constraints on the underlying source
model [ 75 - 77 ]. Considering the ICAMM framework as another approach of SBSS,
some observations are labelled (semi-supervised learning).
Finally, the fourth feature of Mixca is concerned with iterative solutions for
ICAMM. This can be approached as conventional ICA learning for every class,
with a relative degree of correction depending on the conditional probability of
every class to the corresponding observation. Thus, the Mixca algorithm includes a
general scheme, where any ICA algorithm can be incorporated into the learning of
the model.

1.3.2 Learning Hierarchies from ICA Mixtures

A new algorithm to process the parameters (basis vectors and bias terms) learned
from ICA mixtures in order to obtain hierarchical structures is proposed. The
algorithm is agglomerative clustering and uses the symmetric Kullback-Leibler
distance [ 78 ] to select the grouping of the clusters at each level. Meaningful higher
levels of clustering can be obtained, particularly when the classes at the lowest
level fit into an ICA model.
It is well known that local edge detectors can be extracted from natural scene
images by standard ICA algorithms such as InfoMax [ 65 , 79 , 80 ] or FastIca
[ 81 - 83 ] or new approaches such as Linear Multilayer ICA [ 84 ]. In addition, there
is neurophysiological evidence that suggests a relation between the primary visual
cortex activities and the detection of edges. Some theoretical dynamic models of
the feedforward abstraction process from the visual cortex to higher-level

abstraction have been proposed [ 85 ].
The application of the proposed method in image processing has demonstrated
the algorithmic capability for merging similar patches on a natural image; clus-
tering of different images of an object; and the creation suitable hierarchical levels
of clustering from images of different objects. Furthermore the application of the