Hierarchical Clustering from ICA Mixtures - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 96

Fig. 4.7 Two groups of basis functions corresponding to two different objects. The basis
functions at top are from a small box and the basis functions at bottom are from an onion
In all the experiments each image was first converted to greyscale and linearly
normalized so that the pixels had zero mean and unit variance. The first test was to
compare the basis functions of different objects in COIL-100 database. A total of
20 images for each one of eight selected objects were randomly taken from the
database. A total of 2,000 image patches (windows) of 8 9 8 pixels were randomly
taken for each object. From each patch the local mean was subtracted. These data
were used to estimate the basis functions previous to a whitening process using
PCA, with a reduction from 64 features to 40 components. This procedure is
explained in detail in [ 26 , 32 ]. The basis functions were then calculated with the
Mixca algorithm that was performed one time per each object data set in order to
estimate the parameters of three classes. Supervised training and the Laplacian
prior was used in order to estimate the source pdf's. The estimated basis functions
were converted to the original feature space using the dewhitening matrix previ-
ously estimated by PCA. Figure 4.7 shows the 40 basis functions of 8 9 8 cor-
responding to the three class parameters estimated for two of the objects: a box
with an inscribed label (Fig. 4.7 a) and an onion (Fig. 4.7 b). The similarities and
differences between the functions of each object can be observed in this figure: for
instance, the lower frequency in the pattern corresponding to a natural object (the
onion) versus the high frequency in the pattern of an artificial object (the box).
The obtained basis functions were used to measure the distance between classes
by estimating the symmetric Kullback-Leibler (KL) distance from the mixture
matrices previously calculated. The KL distances revealed that basis functions can
find the similarity (short distances) between classes corresponding to the same
object (intra-object), and the difference (long distances) between classes of dif-
ferent objects (inter-object). The KL distances corresponding to the two objects in
Fig. 4.7 are shown in Table 4.2 .
Experiments to create a hierarchical classification of objects were also per-
formed using the data set of the experiment above. The Mixca algorithm was used
to estimate the parameters of eight classes (one class per object). These ICA
parameters build the lowest level of the hierarchy. The intermediate levels of the
hierarchy were then created by applying the agglomerative clustering algorithm
explained in Sect. 4.3.1 . Figure 4.8 shows the hierarchical classification of the