Database Reference
In-Depth Information
resort to numerical or asymptotic methods to obtain an approximation (see
Section 6.5
).
6.4 EM on a Mixture of vMFs (moVMF)
We now consider a mixture of
k
vMF (moVMF) distributions that serves
as a generative model for directional data, and obtain the update equations
for estimating the mixture-density parameters from a given dataset using
the Expectation Maximization (EM) framework. Let
f
h
(
x
|
θ
h
)denoteavMF
distribution with parameters
θ
h
=(
μ
h
,κ
h
)for1
≤
h
≤
k
. Then a mixture of
these
k
vMF distributions has a density given by
k
f
(
x
|
α
h
f
h
(
x
|
θ
h
)
,
Θ) =
(6.6)
h
=1
where Θ =
and the
α
h
are non-negative and sum to
one. To sample a point from this mixture density we choose the
h
-th vMF
randomly with probability
α
h
, and then sample a point (on
{
α
1
,
···
,α
k
,θ
1
,
···
,θ
k
}
d−
1
) following
S
f
h
(
x
|
θ
h
). Let
X
{
x
1
,
···
,
x
n
}
be a dataset of
n
independently sampled
=
Z
{
z
1
,
···
,
z
n
}
points that follow (6.6). Let
be the corresponding set of
hidden random variables that indicate the particular vMF distribution from
which the points are sampled. In particular,
z
i
=
h
if
x
i
is sampled from
f
h
(
x
|θ
h
). Assuming that the values in the set
Z
are known, the log-likelihood
of the observed data is given by
=
n
ln
P
(
X
,
Z|
ln (
α
z
i
f
z
i
(
x
i
|
θ
z
i
))
.
Θ) =
(6.7)
i
=1
Obtaining maximum likelihood estimates for the parameters would have been
easy were the
z
i
truly known. Unfortunately that is not the case, and (6.7)
is really a random variable dependent on the distribution of
—this random
variable is usually called the
complete data log-likelihood
.Foragiven(
Z
X
,
Θ), it
is possible to estimate the most likely conditional distribution of
,
Θ), and
this estimation forms the E-step in an EM framework. Using an EM approach
for maximizing the expectation of (6.7) with the constraints
μ
h
μ
h
=1and
Z|
(
X
Search WWH ::
Custom Search