Machine Recognition in Complex Domain - High Dimensional Neurocomputing: Growth, Appraisal and Applications - page 139

data representation than PCA. The idea is to find transformed images such that the
resultant transformed vectors are statistically independent of each other. The IC rep-
resentation of a face image is estimated on the basis of a set of spatially independent
basis images obtained by ICA. This compressed representation of a face image is a
vector of coefficients and may be used for linearly combining the independent basis
images to generate the face image [ 38 ].
Start with assumption that the face images inX to be a linearmixture of statistically
independent basis images S, combined by an unknown mixing matrix A, such that
X WX. ICA, as an

unsupervised learning algorithm learns weight matrix W, which is used to estimate
a set of independent basis images in the rows of U. W is roughly the inverse matrix
of A. The ICA is carried out in a compressed and whitened principal components
analysis space, where vectors associated small trailing eigenvalues are discarded.
Therefore, before performing ICA, the problem of estimation of W can be simplified
by two preprocessing steps:

= AS. ICA tries to find out the separating matrix W such that U =

• Compression Apply PCA on the image matrix X, to reduce the number of data

to a tractable number, hence reduce the computational complexity. It also makes
method convenient for calculating the representations of considered images. Pre-
applying PCA does not throw away the higher-order relationships, they still exist
but not separated. It enhances ICA performance by discarding the eigenvectors
corresponding to trailing eigenvalues, which tend to capture noise. Empirically it
is seen that including the eigenvectors associated with small eigenvalues will lead
to decreased ICA performance, as they amplify the effect of noise. Let matrix E T
(M by N) contains first M eigenvectors associated with higher eigenvalues of M
face images in its row, then U

WE T .

=

• Whitening Although PCA already removed the covariances in the data but vari-

anceswere not equalized. Therefore, we need to retain the sphering step.Whitening
process transforms the observed vector into a new vector, whose components are
uncorrelated and variances are equalized. The whitening matrix is obtained as
W w = )) − ( 1 / 2 ) . The full transformation matrix will be the product

of whiten (sphere) matrix and matrix learned by ICA. Instead of performing ICA
on the original images, it should be carried out in a compressed and whitened
space, where most of the representative information is preserved.

2 × ( COV ( X

ICA observed wider interest and growing attention after 1990s when various
approaches for ICA techniques were established. Among various prominent tech-
niques for capturing independent components [ 33 , 34 , 37 , 41 , 54 , 55 ], there are
their own strength and weakness. One of the techniques for ICA estimation, inspired
by information theory (infomax), is minimization of mutual information between
random variables. This chapter formulates a complex version of ICA (i.e., C ICA )for
machine recognition using Bell and Sejnowski infomax method in real domain [ 38 ,
41 ]. Infomax performs source separation (to derive independent components) bymin-
imizing the mutual information expressed as a function of higher-order cumulants.
The algorithm for C ICA has been derived from the principle of optimal informa-
tion transfer through complex-valued neurons operated on nonanalytic but bounded