Information Technology Reference
In-Depth Information
evaluate on video to judge if the motion portrayal has become natural during real-
time playback.
In our tests we have doubled frame rates from 25 to 50 fps, which should enable
viewing the video examples in the online material [32] on any modern PC screen
at refresh rates of 50 Hz or 100 Hz, whereas viewing at other rates above 50 Hz
might add some jerkiness from frame repetition (by the graphics/video card and/or
playback software). Comparing the 25 fps input sequences in [32] with the 50 fps
results should however clearly illustrate the difference in quality (e.g. using Virtual-
Dub included in [32]).
As objective measures we have used the mean square error (MSE) and the peak
signal to noise ratio (PSNR). Using the notations given in Sect. 2, the MSE and
PSNR are
PSNR
= 10 log
10
255
2
MSE
1
N
Ω
u
gt
)
2
MSE
=
(
u
−
(9)
where
u
is the frame doubled output and
u
gt
is the ground truth. We sum over all
pixels of the sequence (also the old frames from the input that are not changed in
the output). PSNR is measured relative to the maximum grey value, 255.
3.3
Frame Doubling Results
We generally do not discuss frame repetition results as they are identical to the input.
Thus any description of the input also fits on the corresponding frame doubling
output.
In Fig. 2 results for the sequence
Square
is given.
Square
has 50
×
50 frame
size, is 5 frames long in the input and 9 frames in the output. The 10
10 square
moves diagonally down to the right. The speed of the square is 2 pixels/frame in the
output.
Frame averaging creates a double, semitransparent square as seen in Fig. 2(b).
Variational TSR perfectly recreate the square with GCA on the flow as seen in
Fig. 2(d), but not without GCA (shown in Fig. 2(c)). When watched as video [32],
the square is not perceived as unsharp in the result without GCA and the motion has
become fluent as compared to the input and the result looks identical to TSR with
GCA. The motion in the frame averaging output is jerky and has a clear trail of the
square.
The flows computed by the variational TSR algorithm on
Square
are shown in
Figs. 2(e)-(h). A (dis)occlusion trail can be seen in the flows, which in the non-GCA
version gives some artifacts ( Fig. 2(c)). We also see an nice filling in of the flow (by
the
E
3
-term) in the center of the completely uniform square (image gradient zero,
which gives no flow locally from the BCA and GCA). Even though it is very hard
to detect visually in the flow fields, the flow field of the GCA version is closer to the
correct magnitude and direction at the corners of the square, yielding better intensity
result as seen in Fig. 2(d).
×
