Information Technology Reference
In-Depth Information
The effect of these iterations is to generate sequences of paired distribu-
tions and parameters
q
(
r
)
T
,q
(
r
)
S
}
r∈
N
that satisfy
F
(
q
(
r
+1)
T
q
(
r
+1)
S
,θ
(
r
)
,θ
(
r
+1)
)
{
≥
F
(
q
(
r
)
T
q
(
r
)
S
,θ
(
r
)
). This variant falls in the modified Generalized Alternating
Minimization (GAM) procedures family for which convergence results are
available [6].
We then derive two equivalent expressions of
F
when
q
factorizes as in
˜
D
. Expression (1) of
F
can be rewritten as
F
(
q, θ
)=
E
q
[log
p
(
T
|
S
,
y
,θ
)] +
E
q
[log
p
(
S
,
y
|
θ
)] +
I
[
q
]. Then,
F
(
q
T
q
S
,θ
)=
E
q
T
[
E
q
S
[log
p
(
T
|
S
,
y
,θ
)]] +
E
q
S
[log
p
(
S
,
y
|
θ
)] +
I
[
q
T
q
S
]
=
E
q
T
[
E
q
S
[log
p
(
T
|
S
,
y
,θ
)]] +
I
[
q
T
]+
G
[
q
S
]
,
where
G
[
q
S
]=
E
q
S
[log
p
(
S
,
y
θ
)] +
I
[
q
S
] is an expression that does not depend
on
q
T
. Using the symmetry in
T
and
S
, it is easy to show that similarly,
|
F
(
q
T
q
S
,θ
)=
E
q
S
[
E
q
T
[log
p
(
S
|
T
,
y
,θ
)]] +
E
q
T
[log
p
(
T
,
y
|
θ
)] +
I
[
q
T
q
S
]
T
,
y
,θ
)]] +
I
[
q
S
]+
G
[
q
T
]
,
=
E
q
S
[
E
q
T
[log
p
(
S
|
where
G
[
q
T
]=
E
q
T
[log
p
(
T
,
y
θ
)] +
I
[
q
T
] is an expression that does not depend
on
q
S
. It follows that the E-T and E-S steps reduce to,
|
E-T-step:
q
(
r
)
T
S
,
y
,θ
(
r
)
)]] +
I
[
q
T
]
=arg max
q
T
∈D
T
E
q
T
[
E
[log
p
(
T
|
( )
q
(
r
−
1)
S
E-S-step:
q
(
r
)
S
T
,
y
,θ
(
r
)
)]] +
I
[
q
S
]
=arg max
q
S
∈D
S
E
q
S
[
E
[log
p
(
S
|
(7)
q
(
r
)
T
and the
M-step
θ
(
r
+1)
=argmax
θ∈Θ
y
,
T
,
S
)]
.
More generally, we adopt in addition, an incremental EM approach [6] which
allows re-estimation of the parameters (here
θ
) to be performed based only on
a sub-part of the hidden variables. This means that we incorporate an M-step
(5) in between the updating of
q
T
E
q
(
r
T
q
(
r
)
S
[log
p
(
θ
|
and
q
S
. Similarly, hyperparameters could be
updated there too.
It appears in equations (6), (7) and (5) that for inference the specification
of the three conditional distributions
p
(
t
|
s
,
y
,θ
),
p
(
s
|
t
,
y
,θ
)and
p
(
θ
|
t
,
s
,
y
)is
necessary and sucient.
5.1 Structure and Tissue Conditional E-Steps
Then, steps E-T and E-S have to be further specified by computing the expec-
tations with regards to
q
(
r−
1)
S
and
q
(
r
)
T
. Using the structure conditional model
definition (3), it comes,
Search WWH ::
Custom Search