Graphics Reference
In-Depth Information
an
epipolar plane
. The points where the line joining the two viewpoints
c
0
and
c
1
intersects the image planes are called
epipoles
.
Returning to the problem of pixel correspondence, suppose
p
o
is the pixel at
(
in
I
0
to be matched with a pixel
p
1
in
I
1
.Both
p
0
and
p
1
correspond to point
x
in the scene, but the location of
x
is unknown; it is only known to lie on the ray
−−→
c
0
p
0
, and therefore on the epipolar plane through
c
0
,
p
0
,and
c
1
. The projection
p
1
of
x
in
I
1
therefore must lie on the epipolar line in
I
1
. The neighborhood pixel
search in
I
1
can thus be limited to this epipolar line. This reduces the search space
from two dimensions to one; however, this epipolar optimization can only be used
if the relative positioning of the two cameras is known.
Stereo correspondence by direct searching is relatively simple if the images
are similar, but it breaks down if there is very much noise, or if the cameras that
captured the images were too far apart or had very different orientations. In such
cases manual intervention is likely necessary. Automatic pixel correspondence in
general is still regarded as a difficult problem, even when extensive searching is
applied.
x
,
y
)
5.1.3 Optical Flow
One situation where the primary and secondary images are likely to be similar
enough for automated stereo correspondence arises when the images are succes-
sive frames in a movie or animation. However, the differences in the images may
come from the motion of the objects between frames as well as motion of the cam-
era, so the epipolar approach does not necessarily apply. The general problem of
tracking corresponding points between frames is called
optical flow
.
Optical flow has been extensively studied in computer vision; Beauchemin
and Barron present a survey of optical flow algorithms [Beauchemin and Bar-
ron 95]. The basic problem is to construct a vector field on the primary image
I
0
, in which the vector at each point points to the corresponding point in a sec-
ondary image
I
1
. This vector field is called the
2D motion field
, and optical flow
is sometimes referred to as the computation of
image velocity
. The actual motion
field can be very complicated; in fact, it is not even defined for points that become
visible or invisible between images. The general optical flow problem is to find
an approximation to the true motion field.
For a rasterized image the motion field becomes a discrete vector field, a vec-
tor function
d
of each pixel: if
d
(
x
,
y
)
(
x
,
y
)=(
dx
,
dy
)
,pixel
(
x
,
y
)
in
I
0
corre-
sponds approximately to pixel
(
x
+
dx
,
y
+
dy
)
defining the optical flow vector
d
is to minimize the square of the pixel dif-
ference in a neighborhood of fixed size as in Equation (5.1). That is, the value of
dx
and
dy
for a particular pixel
(
x
,
y
)
(
x
,
y
)
are chosen so that
E
(
dx
,
dy
)
is minimized.
