Information Technology Reference
In-Depth Information
new, interpolated frames. Using a motion compensated variational method, blur will
be the most likely artifact. Thus we need to know (if possible) how much blur is
acceptable to the human visual system.
Judging generally and objectively how unsharp we can allow parts (say every
other frame) of an image sequence to be, is still an open question. Vision research
does not offer an answer as the boundary between sharp and blurred in human per-
ception is still sought for in more limited sub-problems, e.g. in [2] where the authors
try to find out when blur becomes bothersome on simple stationary text/characters.
In [3] (Burr and Morgan) and [4] (Morgan and Benton) it is shown experimentally
that moving objects often appear sharp to the HVS, not because some mechanism
removes blur, but because the HVS is unable to decide whether the object is really
sharp or not. Even though we do not get an answer to our question off blur accep-
tance from vision research, we do get helping pointers: It seems we can allow for
some blur when doing temporal super resolution and still get subjectively good re-
sults (evaluation by the HVS of the viewers). In [5] Chen et al. shows that motion
blur in LCD displays can be reduced by inserting blurred frames between frames
that are enhanced correspondingly in the high frequencies. The safest way towards
optimal TSR is, however, to make the new frames in the output as sharp as possible.
1.3
Related Work
Temporal interpolation of signals is not new, it has been done for a long time for
1D signals in signal processing, but these methods cannot be applied to frame rate
conversion due to the presence of motion.
In medical imaging interpolation of new frames or volumes of a time sequence
of 2D or 3D scans are of interest, mainly in lung (respiratory gated) and heart (heart
gated) imaging. The work by Ehrhardt et al. in [6] is a typical and recent exam-
ple, where temporal super resolution in heart gated imaging is performed using an
accurate flow algorithm, but with simple motion compensated interpolation of in-
tensities along the flow lines to get the new frames. In the field of video processing
there are several TSR patents, e.g. [6, 7, 8], mostly doing flow calculation (good or
bad) followed by some simple, non-iterative averaging along the flow. TSR is also
done in integrated circuits (ICs) as described by de Haan in [9] using 8
8block
matching flow with a median filter for motion compensated interpolation (see [10]
for details). In a recent paper [11] by Dane and Nguyen motion compensated inter-
polation with adaptive weighing to minimize the error from imprecise or unreliable
flow is presented. This elaborate scheme is surely needed as the flow used in [11] is
the MPEG coding vectors, typically prediction error minimizing vectors, which can
be very different from the optical flow.
In [12] Karim et al. focus on improving block matching flow estimation for mo-
tion compensated interpolation in low frame rate video and no less then 16 refer-
ences to other TSR algorithms are given. An overview of early work on motion
compensated temporal interpolation in general (TSR, coding, deinterlacing etc.) is
×
Search WWH ::




Custom Search