Graphics Reference
In-Depth Information
Three-dimensional surveying techniques, which we discuss in Chapter
8
, allow us
to assign a physically correct scale to the reconstructed scene and camera motion
to ensure that all the composited elements are the correct size. Multi-view stereo
techniques, also discussed in Chapter
8
, can be combined with the methods in this
chapter and those in Chapter
5
to create dense 3D reconstructions of objects that can
also be used as inserted scene elements.
While the previous chapters in this topic emphasized 2D image processing, with
little reference to the underlying 3D world from which the images are generated,
this chapter, and the rest of the topic, focuses on processing 3D information. The
fundamental tools we need in this chapter are derived from the pure mathematics of
projective geometry
, since the operation of creating a 2D perspective image of the
3D world is inherently a projective operation. When possible, we try to give a clear
geometric or algebraic explanation of the underlying concepts; however, we cannot
develop the full theory here. Hartley and Zisserman [
188
] and Faugeras and Luong
[
137
] recently produced canonical topics on the theory of projective geometry as it
relates to computer vision, to which we refer the reader for more complete coverage.
The emphasis in this chapter is on recovering the path of the camera in a real-
world, physically accurate coordinate system. All of the methods discussed here
result in a
sparse
reconstruction of scene points in the environment observed by
the cameras; we will show how a camera path can be combined with estimated dense
correspondence fields to create dense three-dimensional models of scene objects
using multi-view stereo algorithms in Chapter
8
.
Webeginby revisiting featurematching and tracking, highlightingparticular issues
related tomatchmoving (Section
6.1
). Then we discuss the key aspects of perspective
image formation, including the way we represent cameras and scene geometry in
3D (Section
6.2
). As a first step, we address the problem of calibrating a single, fixed
camera in a controlled environment — in particular, the determination of internal
parameters using several images of a plane (Section
6.3
). We then extend the dis-
cussion to a rigidly mounted two-camera rig, as might be used for stereo filming
(Section
6.4
).
Next, we discuss the key problem in matchmoving, the structure from motion
scenario in which we reconstruct the unknown path of a camera as it moves freely
through an environment over tens or hundreds of frames (Section
6.5
). This pro-
cess typically includes an initial step where we estimate the geometry of the camera
and scene up to an unknown projective transformation, followed by a step in which
we upgrade the geometry to a reconstruction in which length ratios and angles are
accurate. The results are typically refined by a nonlinear algorithm called
bundle
adjustment
. We also discuss several practical issues that arise when processing long
video sequences in which the camera may traverse a substantial distance. Finally, we
describe modern extensions of matchmoving, including the case of source images
sparsely spread over a wide geographical area (Section
6.6
).
6.1
FEATURE TRACKING FOR MATCHMOVING
The first step in matchmoving is the detection and tracking of features throughout
the image sequence to be processed. Remember that features are regions of an image
