Information Technology Reference
In-Depth Information
motion fields, and
P
2
a term that acts both as likelihood term for the motion field
and as spatiotemporal prior on the image sequence. In the case of frame doubling,
D
consists in “forgetting every second frame”. It becomes a linear map of a bit more
complex form for different frame rate ratios, typically a projection and inverting it
is generally ill-posed.
In this work we do not consider noise contamination between the “ideal”
u
and
the observed
u
0
, i.e. we do not try to denoise
u
0
. The likelihood term
P
0
is then a
Dirac distribution
δ
Du
−
u
0
.
We use a Bayesian to variational rationale a la Mumford [25],
E
(
x
)=
p
(
u
0
|
u
,
D
)=
log
p
(
x
),
to transform our MAP estimation into a continuous variational energy minimization
formulation, taking into account the form of the likelihood term
−
arg.min
(
u
,
v
)
,
Du
=
u
0
E
(
u
,
v
)=
E
1
(
u
s
)+
E
2
(
u
s
,
u
t
,
v
)+
E
3
(
v
)
(2)
(using the same notation for discrete and continuous formulations). Then assuming
some mild regularity assumptions, a minimizing pair (
u
,
v
) must satisfy the condi-
tion
∇
E
(
u
,
v
)=0where
∇
is the gradient, and the solution expressed by the coupled
system of equations
∇
u
E
(
u
,
v
)=0
,
Du
=
u
0
(3)
∇
v
E
(
u
,
v
)=0
.
This system can be considered simultaneous when alternatingly updating the guesses
on solutions to
v
E
= 0 down through the multiresolution pyramid as
discussed in Sect. 1.4. We thus minimize both the flow and intensity energy on each
level of the pyramid as we iterate down through it.
We now discuss the choice of the actual terms in the energy (2). The term
E
0
has already been described above: its function is to preserve input frames unaltered
whenever they are at the position of an output frame.
The term
E
2
is important as it models the consistent temporal transport of infor-
mation into the new frames along the flows (forward and backward). It acts both
as a prior on the intensities and as the likelihood of the motion field. We derive it
from the classical brightness constancy assumption (BCA) which assume intensity
preservation along the motion trajectories. We in fact use its linearized version, the
optic flow constraint (OFC)
∇
u
E
= 0and
∇
∇
u
·
v
+
u
t
= 0, where
∇
denotes the spatial gradient
∂
y
)
t
used first in [26], but we punish a regularized 1-norm form of it, not the
(original) quadratic ones. In the sequel, we write
(
∂
x
,
∇
u
·
v
+
u
t
=
L
v
u
often referred to as the
Lie derivative
of
u
along
v
(although, it should, more cor-
rectly be along the spatiotemporal extension (
v
t
,
1)
t
of
v
).
The term
E
3
is the prior on the flow. It serves the purpose of filling in good
estimates of flow vectors in smooth regions from accurate values calculated where
salient image data is available (edges, corners etc. giving nonzero image gradients).
