Graphics Reference
In-Depth Information
4
Features and Matching
In many visual effects applications, we need to relate images taken from different
perspectives or at different times. For example, we often want to track a point on a
set as a camera moves around during a shot so that a digital creature can be later
inserted at that location. In fact, finding and tracking many such points is critical for
algorithms that automatically estimate the 3D path of a camera as it moves around a
scene, a problem called
matchmoving
that is the subject of Chapter
6
. However, not
every point in the scene is a good choice for tracking, since many points look alike. In
this chapter, we describe the process of automatically detecting regions of an image
that can be reliably located in other images of the same scene; we call these special
regions
features
. Once the features in a given image have been found, we also discuss
the problems of describing, matching, and tracking them in different images of the
same scene.
In addition to their core use for matchmoving, feature detection is also important
for certainalgorithms that estimate dense correspondence between images and video
sequences (Chapter
5
), as well as for bothmarker-based andmarkerless
motion cap-
ture
(Chapter
7
). Outside the domain of visual effects, feature matching and tracking
is commonly used for stitching images together to create panoramas [
72
], localiz-
ing mobile robots [
432
], and quickly finding objects [
456
] or places [
424
] in video
databases.
Feature tracking is a subset of the more general problem of
visual tracking
from
computer vision. However, there are some big differences to keep in mind. Visual
tracking algorithms are usually designed to follow a particular meaningful object
such as a person or car throughout a video sequence. On the other hand, features are
automatically extracted froman imagebasedpurely onmathematical considerations,
and usually look like individually uninteresting blobs or corners. Precise localization
of features is critical for subsequent applications like matchmoving, while a general
visual tracker may use a crude box or ellipse (e.g., [
103
]) to outline the region of
interest. It's also common for general visual trackers to maintain a probabilistic rep-
resentation of an object's state, for example using a Kalman filter (e.g., [
58
]), while
this approach is fairly uncommon in feature tracking. Finally, a major area of interest
in feature matching is the
wide-baseline
case in which the images under considera-
tion were taken from cameras that were physically far apart, whereas visual tracking
generally assumes the camera moves only slightly between images.
While we generally use the term
features
throughout this chapter to denote image
regions of interest, several other terms are often used to describe the same concept,
107
