Biomedical Engineering Reference
In-Depth Information
(
|
)
Let us first derive
p
x
y
. Using arguments similar to those in Sect.
B.6.1
, defining
the Lagrangian as
d
x
d
L[
q
(
x
)
]=
ʸ
q
(
x
)
q
(
ʸ
)
[
log
p
(
x
,
y
,
ʸ
)
−
log
q
(
x
)
−
log
q
(
ʸ
)
]
∞
1
+
ʳ
q
(
x
)
d
x
−
,
(B.72)
−∞
(
)
differentiating it with respect to
q
x
, and setting the derivative to zero, we have
d
(
ʸ
)
log
p
)
+
C
=
ʸ
q
(
x
,
y
,
ʸ
)
−
log
q
(
x
0
,
(B.73)
where
d
C
=−
ʸ
q
(
ʸ
)
log
q
(
ʸ
)
−
1
+
ʳ.
Neglecting
C
, we obtain
d
log
p
,
ʸ
)
,
log
p
(
x
|
y
)
=
ʸ
q
(
ʸ
)
log
p
(
x
,
y
,
ʸ
)
=
E
(
x
,
y
(B.74)
ʸ
where
E
[
·
] indicates computing the mean with respect to the posterior distribution
ʸ
q
(
ʸ
)
. Using exactly the same derivation, we obtain the relationship,
d
x
q
E
x
log
p
,
ʸ
)
,
log
p
(
ʸ
|
y
)
=
(
x
)
log
p
(
x
,
y
,
ʸ
)
=
(
x
,
y
(B.75)
where
E
x
[
·
] indicates computing the mean with respect to the posterior distribution
q
(
x
.
Equations (
B.74
) and (
B.75
) indicate that, to compute the posterior distribution
)
p
(
x
|
y
)
, we need
q
(
ʸ
)
, namely
p
(
ʸ
|
y
)
, and to compute the posterior distribution
p
(
ʸ
|
y
)
,
we need
q
. Therefore, in variational Bayesian inference, we need
an EM-like recursive procedure. We update
(
x
)
, namely
p
(
x
|
y
)
p
(
x
|
y
)
and
p
(
ʸ
|
y
)
by repeatedly applying
Eqs. (
B.74
) and (
B.75
). When computing
p
(
x
|
y
)
using Eq. (
B.74
), we use
p
(
ʸ
|
y
)
(
ʸ
|
)
obtained in the preceding step. When computing
p
y
using Eq. (
B.75
), we use
(
|
)
p
obtained in the preceding step. This recursive algorithm is referred to as the
variational Bayesian EM (VBEM) algorithm.
x
y
References
1. C. M. Bishop,
Pattern recognition and machine learning
. Springer, New York,
2006.
2. H. T. Attius, “A variational Bayesian framework for graphical models,” in
Advances in Neural information processing
, pp. 209-215, MIT Press, 2000.
