ICA and ICAMM Methods - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 39

For the independent components S i ð t Þ , the lagged covariances are all zero due to
independence, without the need for higher-order information to estimate the
model.
The optimization procedure has to minimize the sum of the off-diagonal ele-
ments (diagonalize) of several lagged covariances of s ¼ wz. Considering the
symmetric version C s ð z Þ ¼ 2 C s ð z Þ þ C s ð z Þ

T of the covariance matrix and a set
of chosen lags s denoted by s, the objective function can be written as
X

off W C s ð z Þ W T ð 2 : 23 Þ

s 2 S

The minimization of Eq. ( 2.23 ) can be accomplished by a gradient descent algo-
rithm. Another alternative is to adapt the existing methods for eigenvalue
decomposition for this simultaneous approximate diagonalization of several
matrices. The SOBI algorithm (Second-Order Blind Identification) and TDSEP use
Jacobi-like algorithms for optimization [ 43 , 44 ].
The set of time delays s can be arbitrarily selected or manually given with prior
knowledge. The advantage of second-order methods is their computational sim-
plicity and efficiency. Furthermore, for a reliable estimate of covariances only
comparatively few samples are needed.

2.3 Non-Parametric ICA

The estimation of the densities is, in general, a non-parametric problem. This
means that the number of parameters is infinite, or, in practice, very large. The
non-parametric problems are the most difficult to estimate. As was reviewed in
Sect. 2.1 , most known methods for solving the ICA problem involve specification
of the parametric form of the latent components densities p i and estimation of B
together with parameters of p i using maximum likelihood or minimization of the
empirical versions of various divergence criteria between densities. In practical
applications, the distributions p i of the independent components are generally
unknown, and thus ICA can be considered as a semi-parametric method in which
these distributions are left unspecified.
Conventional ICA techniques have used two methods to avoid non-parametric
estimation. The first method consists of using prior available knowledge about the
densities. The results of the estimator would depend on the specification of the
priors. By including these priors in the likelihood, the likelihood would really be a
function of B only. A second method is to approximate the densities of the
independent components by a family of densities that are specified by a limited
number of parameters. For instance, a simple parameterization of the p i is a single
binary parameters, i.e., the choice between two densities [ 5 ].
Nowadays, there seem to be two research directions in ICA modelling: the first
is motivated to design a signal separation algorithm that is ''truly blind'' to the