Image Processing Reference
In-Depth Information
16.5.2.4
Non-Gaussianity and Negentropy
Another approach is related to the non-Gaussianity of the components and can
be understood if we introduce the central limit theorem. This theorem states
that the linear combination of random variables approaches a Gaussian distri-
bution as more variables are added. From this theorem, we can state that because
the data set is supposed to be a linear mixing of statistically independent random
variables, their linear mixing is supposed to be more Gaussian than the original
ones. When we look for original independent components, we look for a linear
combination of the mixtures . The new variable is maximally non-
Gaussian when it equals one of the independent components. It is now clear
that it is not possible to estimate statistically independent components that are
Gaussian distributed; moreover, higher-order moments of Gaussian-distributed
variables equal zero, so they cannot be estimated by nonlinear decorrelation
methods. In order to estimate the degree of non-Gaussianity of a random
variable, it is possible to use different measures such as kurtosis. However,
negentropy has proven to be more robust, even if computationally expensive
to estimate. Negentropy is defined as
J
(
y
)
s
=Σ
WX
i
i
j
j
H
(
y
) where
H
(y) is the
entropy of the
y
variable and
y
gauss
is a Gaussian variable with the same
covariance matrix as
y
. Because Gaussian variables have the largest entropy
among all variables with equal variance, negentropy is always greater than or
equal to zero. It can be shown that if the estimated independent components
are constrained to have unit variance, estimating the weights
W
such that the new
variables have maximum non-Gaussianity is the same as minimizing their mutual
information. The fastICA algorithm can be used to estimate the independent
components by using of this principle. It is appealing because instead of using
a gradient descent approach to find the solution, it employs a fast fixed-point
iteration scheme. Moreover, the independent components can be found using
a deflationary scheme, which means estimating the independent components
one by one. This operation is simplified if we are working on whitened data
because in the whitened space the directions
w
i
that maximize the non-Gaus-
sianity of are orthogonal. In fact, because the independent components
are uncorrelated it follows that
=
H
(
y
gauss
)
−
wx
i
T
{
}
=
{
{
==
( )
( )
}
T
E
s
i
T
δ
E wx wx
Ewxxw
T
T
TT
j
ij
i
j
i
(16.47)
{}
=
=
w
Exx w
ww
T
T
T
i
j
i
j
I
. The first independent component can be
estimated by calculating the direction
w
1
that maximizes the non-Gaussianity of
. The successive independent components are found as the directions that
maximize the non-Gaussianity of with the constraint that
w
i
lie in the subspace
orthogonal to the one individuated by the directions found in the previous steps.
Another approach consists in estimating the independent component in a single
as for whitened data we have
E
{
xx
T
}
=
wx
i
T
wx
i
T
Search WWH ::
Custom Search