Features and Matching - Computer Vision for Visual Effects - page 108

including interest points , keypoints , and tie points . We generally use the word
matching when discussing an arbitrary pair of images of the same scene, and use
the word tracking when the images come from a video sequence.
This chapter is split into two main sections. We first discuss the key problem of
feature detection — that is, deciding which image regions are sufficiently distinctive
(Section 4.1 ). We then discuss the problem of feature description — that is, decid-
ing how to represent the image information inside each region for later matching
(Section 4.2 ). We briefly describe evaluation techniques that help in deciding on a
good detector/descriptor combination (Section 4.3 ) as well as extensions to color
images (Section 4.4 ).
In this chapter, we generally assume that the problem is to detect and match fea-
ture points in a set of natural images, e.g., acquired from a camera on location. This
is frequently the situation for matchmoving with a freely moving camera, as we'll
discuss in Chapter 6 . When we have more control over the environment — for exam-
ple, a soundstage set — it's common to introduce artificial tracking markers (e.g.,
gaffer-tape crosses on the surfaces of a blue- or green-screen set) that are relatively
straightforward to detect and track. We discuss the problem of designing distinctive
artificial tracking markers in Section 4.5 .

4.1 FEATURE DETECTORS

Initially, we'll assume that a feature is a square block of pixels centered at a certain
location in an image. Our first goal is to mathematically characterize what makes a
good feature. Intuitively, we want to select a block that is highly distinctive, so that in
a different image of the same scene, we can find a unique match. Put another way,
we want the detection to be repeatable — that is, given a different image of the same
scene, the feature is distinctive enough that we canfind it again in the correct location.
Figure 4.1 illustrates several feature candidates in an example image. Candidate A
is a poor choice of feature, since this nearly-constant-intensity patch is almost iden-
tical to other nearly-constant-intensity patches in the image. Candidate B is a better
feature, since the strong edge passing through it makes it more distinctive. However,
there are still several blocks in the image that are almost identical to Candidate B,
which can be obtained by sliding the block along the edge; this ambiguity is called
the aperture problem . Candidates C and D are good choices for features; the image
intensities at Candidate C form a corner and those at Candidate D form a blob ; both
blocks are locally unique. That is, each block does not resemble any other block in
its local neighborhood. In the following sections, we formalize this intuition that a
feature should have locally distinctive intensities, and discuss detectors or interest
operators that automatically find such features. In this section, we'll assume that
the images under consideration are grayscale, and will discuss extensions to color in
Section 4.4 .

4.1.1 Harris Corners

Moravec [ 335 ] was among the first to observe that the cornerness of a block of pixels
could be quantified by comparing it to adjacent blocks in the horizontal, vertical, and