One-ICA Version of the Mixca Algorithm - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 178

Appendix
One-ICA Version of the Mixca Algorithm

ICA defines a generative model for the observed data. The conventional model is
given by:

x ¼ As ð A : 1 Þ

where x is the observed data as a random vector whose elements are the mixtures
x 1 ; ... ; x N ; A is a NxM matrix, called the mixing matrix, and s is a random vector
with M elements s 1 ; ... ; s M ; called the source elements.
The goal of ICA is to find a linear transformation of the data such that the latent
variables are as statistically independent from each other as possible. Suppose y is
an estimate of the sources, so that:

y ¼ Wx

ð A : 2 Þ

When the sources are exactly recovered, W is the inverse of A.
The probability density function of the data x can be expressed as:

p ð x Þ¼j det W j p ð y Þ ð A : 3 Þ

where p(y) can be expressed as the product of the marginal distributions since it is
the estimate of the independent components:

p ð y Þ¼ Y

M

p i ð y i Þ ð A : 4 Þ

i ¼ 1

If we assume a non-parametric model for p(y) we can estimate the source pdf's
from a set of training samples obtained from the original dataset using equation
( A.2 ). The marginal distribution of a reconstructed component is approximated as
(kernel density estimation):
p ð y m Þ¼ a X
n 0

y m y ð n 0 Þ
m

2

h

e

2 ; m ¼ 1...M

ð A : 5 Þ