Graphics Reference
In-Depth Information
to be related by a global, parametric transformation, so that
(
u
,
v
)
is a closed-form
function of
. In this case, the problem is called
registration
, and we discuss
estimation methods when the transformation is affine or projective (Section
5.1
).
The next class of methods assumes that a set of sparse feature matches is available
(e.g., obtained using any of the methods in the previous chapter) and that the dense
correspondence field smoothly interpolates these matches. We discuss this problem
of
scattered data interpolation
in Section
5.2
.
The methods in these sections implicitly assume that every point in the first image
has a match in the second image, and vice versa, so that the dense correspondence
is like a “rubber sheet” that warps the coordinate system of the first image to that of
the second. However, these approaches do not allow for the possibility of
occlusions
,
regions visible in one image but not the other, or
discontinuities
, regions where
the
(
x
,
y
)
(
u
,
v
)
field does not smoothly vary, both of which are common in real-world
scenes.
Most generally, we consider a pair of images of the same scene at different times
taken from different perspectives, which could contain multiple moving objects.
The dense correspondence field resulting from changes in apparent brightness is
typically called the
optical flow field
. We discuss this well-studied problem, from
early, now-classical methods to more modern approaches (Section
5.3
). Initial work
assumed that the different perspectives were relatively close together spatially and
that there were no occlusions or discontinuities. However, robust hierarchical meth-
ods and other extensions now permit dense correspondence to be estimated when
the cameras are relatively far apart.
When the images are known to be taken close together and the scene has not
changed (e.g., by two synchronized cameras mounted on a rigid rig), the special
case of the optical flow problem is called
stereo correspondence
. This configuration
forces the correspondence of each pixel in one image to lie on a straight line in the
other image, greatly reducing the search space for the motion vectors. We introduce
the
epipolar geometry
for an image pair andmethods for its estimation using a set of
featurematches (Section
5.4
). We also discuss how to
rectify
an image pair to ease the
search for correspondences along pairs of epipolar lines. We then overview the stereo
correspondence problem, an area of major interest in computer vision (Section
5.5
).
Since themotion vectors in the stereo problemare discretized to a small set of values,
we can apply modern optimization tools like graph cuts and belief propagation to
efficiently solve the problem.
The rest of the chapter addresses visual effects applications of a dense correspon-
dence field once it has been obtained. We first discuss
video matching
, the problem
of aligning two sequences that follow roughly the same camera trajectory but at dif-
ferent speeds (Section
5.6
). Video matching is the central problem for effects shots
in which multiple camera passes of different elements need to be composited into a
final result. Next, we discuss
imagemorphing
, the visually compelling effect of trans-
forming one object into another (Section
5.7
). In this case, we need to create a dense
correspondence field between two fundamentally
different
scenes, so scattered data
interpolation methods are typically used instead of optical flow and stereo methods.
Finally, we discuss the problemof
view synthesis
, where the goal is to generate phys-
ically realistic “in-between” views of a scene from different perspectives than either
of the source images (Section
5.8
).
