Graphics Reference
In-Depth Information
4.3
EVALUATING DETECTORS AND DESCRIPTORS
Given thewide range of available detectors and descriptors discussed so far, choosing
an appropriate combination for a given problem seems a daunting task. Luckily, sev-
eral research groups have undertaken thorough analyses of detectors and descriptors
both separately and together. We briefly summarize several findings here.
The main figure of merit for evaluating a feature detector is its
repeatability
. That
is, a feature detected in one image should also be detected in the correct location
in an image of the same scene under different imaging conditions. These different
conditions could include changes in camera settings (such as focal length, blur, or
image compression), camera viewpoint (inducing affine or more complex changes in
pixel patterns), or the environment (such as global or local illumination).
Studies of detector repeatability [
431
,
329
] first focused on images of a planar sur-
face, like a graffiti-covered wall, taken from different perspectives. Since all points in
suchan image pair are relatedby the same projective transformation (see Section
5.1
),
we can immediately determine the correct location in the second image of any feature
point detected in the first image, and check to see if any features were detected near
this location. Figure
4.21
a illustrates the idea. The
repeatability score
of a detector
for a given image pair is then
R
RS
=
(4.41)
min
(
N
1
,
N
2
)
where
N
1
and
N
2
are the numbers of features detected in each image respectively,
and
R
is the number of repeated detections. An ideal detector has a repeatability of 1.
If the feature detector also produces an estimated support region (e.g., an affine-
covariant ellipse), these regions should also transform appropriately under different
p'
q
p
p'
q
(a)
(b)
Figure 4.21.
Evaluating feature detector repeatability. (a) A feature is detected at location
p
in
the left image. If two images are related by a projective transformation, we can easily determine
the location
p
corresponding to the same position in the right image. If a feature
q
is detected
within a small neighborhood of
p
in the second image, we say the detection is repeated. (b) If
a feature detector produces a support region, we can compare them for a more stringent test of
repeatability. A detection is repeated if the ratio of the intersection of the two regions to their
union is sufficiently large.
