Graphics Reference
In-Depth Information
5.5
STEREO CORRESPONDENCE
Suppose that the two images
I
1
and
I
2
for which we want to compute dense corre-
spondence are taken at the same time instant by two cameras in different positions.
Thus, we can estimate the epipolar geometry and rectify the image pair, as described
in the previous section. In this case, the dense correspondence problem reduces to
the
stereo correspondence
problem, one of the most well-studied problems in com-
puter vision. We can think of stereo correspondence as a special case of optical flow,
with a few key differences:
•
The dense correspondence field is specified by one number at each pixel,
the
disparity
. The disparity is simply defined as
x
x
for a correspondence
−
x
,
y
)
}
{
(
x
,
y
)
,
(
y
since the images are already rectified. An
image
d
(
x
,
y
)
formed from estimates of the disparity
d
at each point
(
x
,
y
)
∈
I
1
is called a
disparity map
for
I
1
.
•
Unlike optical flow, where
takes on continuous values, the disparity
values in stereo are conventionally
quantized
, most frequently into units of
pixels. The physical configuration of the cameras also introduces a maximum
range of possible disparity values. The set of possible disparity values being
discrete and finite allows us to use powerful, efficient optimization methods
based on graph cuts and belief propagation.
(
u
,
v
)
•
Occlusions
in the scene are often explicitly modeled in stereo algorithms as
opposed to implicitly modeled as in optical flow.
•
In addition to an analogue of the smoothness assumption from optical
flow, stereo algorithms sometimes (but not always) introduce a
monotonic-
ity
assumption stating that corresponding points appear in the same order
along matching scanlines. This simplifies the problem even further at the
risk of poorly modeling certain real-world scenes containing thin objects (see
Section
17
).
•
Stereo algorithms sometimes (but not always) assume that additional
calibra-
tion
information is known about the cameras that acquired the images. This
camera calibration results in immediate, high-quality estimates of the epipo-
lar geometry and maximum disparity range. However, feature matches can
typically be used to estimate the epipolar geometry to set up a stereo problem,
as described in the previous section.
•
We must always remember that stereo algorithms assume that the images are
acquired
simultaneously
, which means there can be no non-rigid motion of
the scene or independent motion of objects between the two images.
We now discuss the main approaches to the stereo correspondence problem,
which we will see have a very different character than methods for optical flow. In
their overview of optical flow algorithms, Baker et al. [
27
] noted that the current top
16
We conventionally assume
I
1
is to the left of
I
2
with respect to the scene. Therefore, a point in
I
1
should appear to be further to the right than its matching position in
I
2
(see Figure
5.11
b), and
we define the disparity to be the positive number
x
x
. Note that we're assuming a Cartesian
coordinate system for pixels (i.e., the
x
axis is horizontal and the
y
axis is vertical).
−
