Information Technology Reference
In-Depth Information
The first expression can be evaluated by combining (7.6) and (7.7) to get
E
W,τ
(ln
p
(
Y
|
W
,
τ
,
Z
))
=
n
z
nk
j
w
kj
x
n
,τ
−
1
E
W,τ
(ln
N
(
y
nj
|
))
k
k
2
ln 2
π
+
n
=
n
z
nk
j
z
nk
j
1
1
2
E
τ
(ln
τ
k
)
−
k
k
2
n
z
nk
j
E
W,τ
τ
k
(
y
nj
−
w
kj
x
n
)
2
1
−
k
D
2
=
z
nk
E
τ
(ln
τ
k
)
n
k
2
n
z
nk
j
E
W,τ
τ
k
(
y
nj
−
w
kj
x
n
)
2
+const.
,
1
−
(7.59)
k
where
k
z
nk
= 1 was used. Using (7.12) and (7.11), the second expectation
results in
V
)) =
n
E
V
(ln
p
(
Z
|
z
nk
E
V
(ln
g
k
(
x
n
))
k
≈
z
nk
ln
g
k
(
x
)
|
v
k
=
v
k
,
(7.60)
n
k
where the expectation of ln
g
k
(
x
n
) was approximated by the logarithm of its
maximum a-posteriori estimate, that is, ln
g
k
(
x
n
) evaluated at
v
k
=
v
k
.This
approximation was applied as a direct evaluation of the expectation does not
yield a closed-form solution. The same approximation was applied by Waterhouse
et al. [227, 226] for the MoE model.
Combining the above expectations results in the posterior
ln
q
Z
(
Z
)=
n
z
nk
ln
ρ
nk
+const.
,
(7.61)
k
with
2
j
E
W,τ
τ
k
(
y
nj
−
w
kj
x
n
)
2
.
|
v
k
=
v
k
+
D
2
E
τ
(ln
τ
k
)
1
ln
ρ
nk
=ln
g
k
(
x
n
)
−
(7.62)
Without the logarithm, the posterior becomes
q
Z
(
Z
)
∝
n
k
ρ
z
nk
nk
, and thus,
under the constraint
k
z
nk
=1,weget
q
Z
(
Z
)=
n
ρ
nk
j
ρ
nj
r
z
nk
nk
,
with
r
nk
=
=
E
Z
(
z
nk
)
.
(7.63)
k
As for all posteriors, the variational posterior for the latent variables has the
same distribution form as its prior (7.12).
Search WWH ::
Custom Search