Biomedical Engineering Reference
In-Depth Information
(
,
ʸ
|
)
In order to compute the posterior distribution
p
x
y
, we must compute the integral
∞
Z
=
p
(
y
|
x
,
ʸ
)
p
(
x
)
p
(
ʸ
)
d
x
d
ʸ
.
−∞
However, in general, this integral
Z
does not have a closed-form solution, and it is
hard to compute the posterior distribution
p
using Bayes' rule, i.e., Eq. (
B.66
).
In Sect.
B.6.1
, the posterior distribution is obtained using the optimization of the
factional called the free energy, and not using Bayes' rule.We here use this variational
technique to obtain the posterior distribution. The free energy is expressed as
(
x
,
ʸ
|
y
)
d
F
[
(
,
ʸ
),
ʸ
]=
ʸ
(
,
ʸ
)
[
(
,
,
ʸ
)
−
(
,
ʸ
)
]
.
q
x
d
x
q
x
log
p
x
y
log
q
x
(B.67)
We use an approximation that the joint posterior distribution is factorized, i.e.,
(
,
ʸ
)
=
(
)
(
ʸ
),
q
x
q
x
q
(B.68)
which is called the variational approximation. Using this approximation, the free
energy is given by
d
F
[
q
(
x
),
q
(
ʸ
),
ʸ
]=
ʸ
d
x
q
(
x
)
q
(
ʸ
)
[
log
p
(
x
,
y
,
ʸ
)
−
log
q
(
x
)
−
log
q
(
ʸ
)
]
.
(B.69)
(
|
)
(
ʸ
|
)
are derived by
jointly maximizing the free energy in Eq. (
B.69
). That is, the estimated posterior
distributions,
The best estimate of the posterior distributions
p
x
y
and
p
y
p
(
x
|
y
)
and
p
(
ʸ
|
y
)
,aregivenby
p
(
x
|
y
),
p
(
ʸ
|
y
)
=
argmax
q
F
[
q
(
x
),
q
(
ʸ
),
ʸ
]
,
(
x
),
q
(
ʸ
)
subject to
∞
−∞
and
∞
−∞
d
x
q
(
x
)
=
1
,
d
ʸ
q
(
ʸ
)
=
1
.
(B.70)
Therefore, according to Eq. (
B.64
), the estimated posterior distributions
p
(
x
|
y
)
and
p
(
ʸ
|
y
)
satisfy the relationship:
K
L
p
)
,
p
(
x
|
y
),
p
(
ʸ
|
y
)
=
argmin
q
(
x
,
ʸ
|
y
)
||
q
(
x
,
ʸ
|
y
(
x
),
q
(
ʸ
)
where
q
(
x
,
ʸ
|
y
)
=
q
(
x
|
y
)
q
(
ʸ
|
y
).
(B.71)
obtained by max-
imizing the free energy jointly minimizes the KL distance between the true and
approximated posterior distributions. In other words, the estimated joint posterior
Namely, the estimated posterior distributions
p
(
x
|
y
)
and
p
(
ʸ
|
y
)
p
is the “best” estimate in the sense that it minimizes the KL
distance from the true posterior distribution.
(
x
,
ʸ
|
y
)
=
p
(
x
|
y
)
p
(
ʸ
|
y
)
