Information Technology Reference
In-Depth Information
activation function [ 56 ]. The main emphasis is to compare existing ICA or R ICA
algorithm with C ICA for feature extraction in image database. Before summarizing
infomax in real and complex domain it is necessary to briefly examine its derivation.
Mutual information is a natural measure of independence between random vari-
ables (sources). Infomax algorithm [ 35 , 41 ] uses it as a criterion for finding the ICA
transformation. All common ICA algorithms iteratively optimize a smooth function
(Entropy), whose global optima occurs when the output vectors ( u
U ) are indepen-
dent. Finding independent signals by maximizing entropy is known as infomax [ 41 ].
Given a set of signal mixture X and a set of identical model cumulative distribution
function (cdf) f of source signals (independent components). ICAworks by adjusting
the unmixing coefficient W, in order to maximize the joint entropy of the distribution
of Y
or equivalently stating that probability density function (pdf) of
Y is uniform. When W is optimal or Y has maximum entropy then extracted signals
or basis images in U are independent.
=
f
(
U
=
WX
)
7.2.4 Entropy and ICA Gradient Ascent
Entropy is defined as an average amount of certainty associatedwith a series of events.
Less certainty corresponds to higher entropy hence uniform distribution of variables.
For a set of signal mixture X consider the entropy of vector variable Y
=
f
(
U
)
where
=
U
WX is the set of signals extracted by the unmixing matrix W. For a finite set of
values sampled from a distribution with joint pdf p Y (
)
Y
of Y, the entropy of Y can
be defined as
H
(
Y
) =−
p Y
(
Y
)
ln p Y
(
Y
)
dY
(7.6)
The joint entropy is essentially a measure of uniformity of a multivariate pdf
p Y (
T be matrix of unknown basis images (source
signals). They have common model cdf f and model pdf p s ( p s
Y
)
of Y. Let S
= (
s 1 ,
s 2 ,...
s M )
f ). If joint
=
entropy of Y
is maximum then pdf p u of each extracted signal/image in U
will match the pdf p s of source signal/images. If an unmixing matrix W exists which
extracts signals/images in U from a set of image mixture X, then entropy of the Y is
given as:
=
f
(
U
)
M
M
H
(
Y
) =
ln p s (
U i )
+
ln
|
W
| −
ln p x (
X i )
(7.7)
i =
1
i =
1
where
stands for expected value. The last term in Eq. 7.7 is the entropy H(X) of the
set of mixtures x. The unmixing matrixW that maximize H(Y) does so irrespective of
the pdf p X (
.
defines the entropy H(X) of the mixtures of x which
can not be effected by W. Thus from general definition one may deduce that the last
term (H(X)) in Eq. 7.7 can be ignored when seeking for an optimal W that maximizes
X
)
, because p X (
X
)
 
Search WWH ::




Custom Search