Information Technology Reference
In-Depth Information
in [ 17 ]. In this model, the form of the latent variable model is closely related to
probabilistic principal component analysis (PPCA) [ 18 , 19 ]. The construction of
the hierarchical tree proceeds top-down. At the top level of the hierarchy, a single
visualization plot corresponding to a single model is defined. This model is par-
titioned into ''clusters'' at the second level of the hierarchy considering a proba-
bilistic mixture of latent variable models. Subsequent levels, which are obtained
using nested mixture representations, provide successively refined models of the
data set [ 17 ]. ICA model-based hierarchies have also been explored. For instance,
in [ 20 ], a method for capturing nonlinear dependencies in natural images for image
segmentation and denoising is presented. It makes use of lower level linear ICA
representation and a subsequent mixture of Laplacian distributions for learning the
nonlinear dependencies.
The method proposed in this chapter corresponds to hierarchical clustering of
agglomerative type. It starts from a set of ICA mixture parameters that are
extracted from the data using a learning process as explained in Chap. 3 . Each
cluster at the first level of the hierarchy is characterized with the parameters of a
single ICA model. These parameters (mixture matrices, bias vectors, and source
probability density functions) are used to estimate the proximities between clusters
pairwise using the Kullback-Leibler distance [ 21 ]. The pdf of the sources is
estimated using a non-parametric kernel-based density. During the merging of the
clusters, the entropy and cross-entropy of the sources have to be estimated. This
cannot be obtained analytically, and thus an iterative suboptimal approach is
applied using a numerical approximation from the training data.
The structure of several ICA subspaces at the bottom level of the hierarchy
allows non-gaussian mixtures to be modelled. The independence relations between
the hidden variables at this lowest level are relaxed at higher levels of the hier-
archy allowing more flexible modelling. The subspaces constructed by the method
at intermediate levels of the hierarchy represent different degrees of dependence of
the variables. Thus, this work can be related to tree-dependent component analysis
(TCA), which finds ''clusters'' of components such that the components are
dependent within a cluster and independent between clusters [ 22 ]. Topographic
independent component analysis (TICA) is another method that considers the
residual dependence after ICA. This method defines a distance between two
components using higher-order correlations, and it is used to create a topographic
representation [ 23 ].
The clustering procedure was tested with simulations and two applications. The
simulations considered ICA mixtures of diverse kinds of densities: uniform,
K-type [ 24 ], Laplacian, and Rayleigh. The quality of the clustering (selection of
number of clusters in the dendrogram) was tested with the partition and partition
entropy coefficients [ 25 ]. There are many practical applications of statistical
learning where it is useful to characterize data hierarchically. We selected real
image processing since it is classical and the results can be easily interpreted. The
objectives were real object recognition and image segmentation.
Search WWH ::




Custom Search