Graphics Reference
In-Depth Information
just a few pixels, the offset vectors are computed directly. At the next finer level,
the computed vector of each pixel is copied to the corresponding block of four
pixels in the finer level (see the very left of
Figure 5.5
)
. The vector field at the
finer level is then a matter of updating these offsets. The process stops at the finest
level, which corresponds to the original images. Image pyramids are particularly
effective when the motion between frames is relatively large. However, the exis-
tence of occlusion causes difficulty, regardless of the method employed, because
pixels slip in and out of view. Discontinuity in general, whether it arises from
discontinuous motion or discontinuous changes in lighting, is a serious problem
for optical flow computation.
Optical flow determination has a number of applications. Video compression
is one. If the optical flow changes little between several sequential frames, the
middle frames can be reconstructed by interpolating pixels from the offset vec-
tors of neighboring frames, so the middle frames need not be stored. Some con-
temporary high-end digital television systems do something similar, employing
a kind of inverse of this approach to create smoother motion. These televisions
are capable of displaying frames at a higher rate than video frames are delivered.
Approximate frames in between true frames can be constructed by interpolating
the motion field. Inserting such frames can cause motion to appear smoother,
although the effectiveness of this technique depends on the nature of the video.
5.1.4 Refined Version of Stereo Correspondence
Methods for stereo correspondence traditionally assume that the images are sim-
ilar. The paper entitled “Video Matching” by Peter Sand and Seth Teller intro-
duced a technique that works for images having substantially different appear-
ances [Sand and Teller 04]. The goal of the research described in this paper is
how to obtain correspondences between two similar video sequences captured
separately, albeit from similar viewpoints. Two sequences of a similar environ-
ment captured at different times may have differences for several reasons: the
lighting conditions might be different, the scene may have some minor changes,
such as added or removed elements, and the camera positions are not likely to be
identical. The two images at the top of Figure 5.6 provide an example. The teapot
in the left image is missing from right image; furthermore, the camera position
has changed slightly between images. A human observer might see these images
as quite similar, but the variation in the individual pixels is significant enough to
cause trouble for automatic pixel correspondence.
Sand and Teller's approach to image correspondence considers two separate
aspects: a measure of pixel matching consistency, and a measure of motion con-
sistency. Pixel matching is done in a manner similar to the basic stereo correspon-
