Hierarchical Clustering from ICA Mixtures - On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling - page 85

4.2 Problem Statement and Distance Between ICA Clusters

The conditional probability density of an observation vector X for cluster C k ; k ¼
1 ; 2 ; ... ; K h þ 1 in layer h ¼ 1 ; 2 ; ... ; K is p x = C k

: At the first level, h ¼ 1 ; it

: is:

is modelled by the K-ICA mixtures, i.e., p x = C k

¼ det A 1
k

p s ðÞ; s k ¼ A 1

p x = C k ð x b k Þ ð 4 : 2 Þ

k

At each consecutive level, two clusters are merged according to some minimum
distance measure until only one cluster is reached at level h ¼ K :
For the distance measure, we use the symmetric Kullback-Leibler divergence
between the ICA mixtures, which is defined for the clusters u ; v by:

¼ Z p x = C u

dx þ Z p x = C v

log p x = C u log p x = C v

D KL C u ; C v

dx

ð 4 : 3 Þ

p x = C v p x = C u

For layer j ¼ 1 ; from Eq. ( 4.3 ), we can obtain the following:

D KL C u ; C v ð Þ D KL p x u ðÞ== p x v ðð Þ

¼ Z p x u ðÞ log p x u ðÞ

p x v ðÞ dx þ Z p x v ðÞ log p x v ðÞ p x u ðÞ dx

ð 4 : 4 Þ

For brevity, we write p x ðÞ¼ p x = C u and omit the superscript h ¼ 1 : For
simplicity, we impose the independence hypothesis and we suppose that both
clusters have the same number of sources M:

Q

M

p s u i s u ðÞ

i ¼ 1

s u i ¼ A 1 p x u ðÞ¼ ; ð X b u i Þ

u i

j det A u j

ð 4 : 5 Þ

s v j

Q

M

p s v j

j ¼ 1

s v i ¼ A 1

p x v ðÞ¼ ; ð X b v i Þ

v i

j det A v j

4.3 Merging ICA Clusters with Kernel-Based Source
Densities

The pdf of the sources is approximated by a non-parametric kernel-based density
for both clusters:

2 2

s u ðÞ¼ X s v j ¼ X

N N

s v j s v j ðÞ
h

s u i s u i ðÞ
h

ae 2 ae 2 p s u i ; p s v j : ð 4 : 6 Þ

n ¼ 1 n ¼ 1