Database Reference
In-Depth Information
variant that could adapt in an on-line fashion leading to balanced cluster-
ing solutions was developed by (7). Balancing was encouraged by taking
a frequency-sensitive competitive learning approach in which the concentra-
tion of a mixture component was made inversely proportional to the number
of data points already allocated to it. Another online competitive learning
scheme using vMF distributions for minimizing a KL-divergence based distor-
tion was proposed by (43). Note that the full EM solution was not obtained
or employed in either of these works. Recently a detailed empirical study of
several generative models for document clustering, including a simple movMF
model that constrains the concentration
κ
to be the same for all mixture com-
ponents during any iteration, was presented by (56). Even with this restric-
tion, this model was superior to both hard and soft versions of multivariate
Bernoulli and multinomial models. In recent years, the movMF model has
been successfully applied to text mining and anomaly detection applications
for the NASA Aviation Safety Reporting System (ASRS) (47; 46).
Recently, (10) discussed the modeling of high dimensional directional data
using mixtures of Watson distributions, mainly to handle axial symmetries in
the data. The authors of (10) followed the parameter estimation techniques
developed in this chapter to obtain numerical estimates for the concentration
parameter
κ
for Watson distributions. Additionally, alternate parameter esti-
mates along with a connection of mixture of Watson based models to
diametric
clustering
(19) were developed in (45). For text data, mixtures of Watson dis-
tributions usually perform inferior to moVMF based models, though for gene
expression data they could be potentially better.
6.3 Preliminaries
In this section, we review the von Mises-Fisher distribution and maximum
likelihood estimation of its parameters from independent samples.
6.3.1 The von Mises-Fisher (vMF) Distribution
A
d
-dimensional unit random vector
x
(i.e.,
x
∈
R
d
and
x
=1,orequiva-
lently
x
d−
1
)issaidtohave
d
-variate von Mises-Fisher (vMF) distribution
if its probability density function is given by
∈
S
μ, κ
)=
c
d
(
κ
)
e
κμ
T
x
,
f
(
x
|
(6.1)
where
μ
=1,
κ
≥
0and
d
≥
2. The normalizing constant
c
d
(
κ
)isgivenby
κ
d/
2
−
1
(2
π
)
d/
2
I
d/
2
−
1
(
κ
)
c
d
(
κ
)=
,
(6.2)
Search WWH ::
Custom Search