Independent Component Analysis and Blind Source Separation - Biomedical Signal Analysis: Contemporary Methods and Applications - page 133

For this, develop det W by the i -th row to get

n

1) i+k w ik det W (ik) . det W = ( −

k=1

Then, taking derivative by w ij shows the claim.

C ∞ ( Lemma 4.2: For W ∈ Mat( n × n ; R )and p i ∈ R , R ), i =1 ,...,k

n

∂
∂ W

ln p i ( Wx ) i = g ( Wx ) x ,

i=1

for x ∈ R

n ,wherefor y n ,

g ( y ):= p i ( y i )
p i ( y i )

∈ R

n
i=1 ∈ R

n .

Proof

We have to show that

n

ln p k ( Wx ) k = p i ( y i )

∂
∂w ij p i ( y i ) x j

k=1

This follows directly from the chain rule.

Bell-Sejnowski algorithm
With the following algorithm, Bell and Sejnowski gave one of the first
easily applicable ICA algorithms [25]. It maximizes the likelihood from
above by using gradient ascent.
The goal is to maximize the likelihood (or equivalently the log
likelihood) of the parametric ICA model. If we assume that the source
densities are differentiable, we can do this locally, using gradient ascent.
The Euclidean gradient of the log likelihood can be calculated, using
lemmata 4.1 and 4.2, to be

1
T

∂ ln L ( B )
∂ B

= B − + E ( g ( Bx ) x )

with the n -dimensional score function g = g 1 × ... × g n .Thusthelocal

update algorithm goes as follows.
Algorithm: ( gradient ascent maximum likelihood )Choose η> 0and