Information Technology Reference
In-Depth Information
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
Fig. 2
Frame doubling of the 50
×
50 sequence
Square
.
a
Original frame 3 (5 in the output).
The new frame 6 of the output created by
b
frame averaging,
c
variational TSR without GCA,
and
d
variational TSR with GCA. Optic flows computed by variational TSR:
e
backward from
frame 6 to 5 without GCA,
f
forward from frame 6 to 7 without GCA,
g
backward from frame
6to5withGCA,and
h
forward from frame 6 to 7 with GCA. In this color representation, the
hue value gives the flow direction as coded on the boundary and the intensity gives the flow
magnitude (normalized in [0.5-1]).
The sequence
Cameraman Pan
is a pan (10 pixel/frame in the input) across
the image
Cameraman
in a 130
100 window. On the sequence
Square
, frame
averaging was performing bad but not unacceptable. On
Cameraman Pan
the per-
formance of frame averaging is unacceptably bad as Fig. 3(b) illustrates. Artifacts
this bad are clearly visible when viewing the result as video [32] and the motion
seems, if possible, even more jerky than the motion in the 25 fps input. The motion
of the two variational TSR outputs are much smoother and appears very natural. In
Figs. 3(c) and 3(d) it is also seen how the new frames produced with variational
TSR are very similar to the original frame shown in Fig. 3(a). The only difference
is a slight smoothing, which is only seen in the stills.
Some minor (dis)occlusion errors occur at the frame boundaries of
Cameraman
Pan
when details leave or enter the frame, which are only spotted during video
playback if one looks for them, or happens to focus on that particular part of the
frame. The cause of this problem is our use of Neumann boundary condition: When
a flow vector points out of the frame, we use Neumann BC to spatially find a re-
placement value in side the frame. This creates a high magnitude temporal gradient
if the replacement pixel (in the neighboring frame) is very different from the pixel
currently being processed, resulting in increased spatial diffusion (which is often
unreliable).
The sequence
Building
is a 284
×
236 cutout of a PAL DVD (telecined from
film). The scene chosen has a camera tilt down a building with discrepancies from
×
