Information Technology Reference
In-Depth Information
Ta b l e 2 .
Accuracy and Macro
F
1
for the local feature based classifiers
Φ
m
and for the
kNN
classifiers based on the various image similarity measures proposed for best
k
and related to the
and
version
m
classifier
s
s
1, and
s
m, and
s
ı, and
s
h, and
similarity
.94
.85
.90
.91
.93
SIFT
Accuracy
SURF
.93
.80
.88
.87
.92
.94
.84
.89
.91
.93
SIFT
F
1
Macro
SURF
.91
.78
.87
.86
.89
In Figure 3 we report the performance obtained by using the
Percentage of Matches
classifier, i.e., the image similarity based classifier
Φ
s
using the similarity measure
s
m
.
For each distance ratio threshold
c
we report the best result obtained for
k
between 0
and 100. As mentioned in Section 4.2, in the paper where SIFT [14] was presented,
Lowe suggested to use
0
.
8
as distance ratio threshold (
c
). The results confirm that the
threshold proposed in [14] is the best for both SIFT and SURF and that the algorithm is
stable around this values. In Table 1, results were reported for
s
m
and
s
h
with
c
=0
.
8
for both SIFT and SURF.
Let us now consider the confidence
ν
doc
assigned to the predicted label of each image
(see Section 4.1). This confidence can be used to obtain greater
accuracy
at the price
of a certain number of false dismissals. In fact, a confidence threshold can be used to
filter all the label assigned to an image with a confidence
ν
doc
less than the threshold.
In Figure 4 we report the
accuracy
obtained by the
s
h,and
measure using SIFT, varying
the confidence threshold between 0 and 1. We also report the percentage of images in
Te
that were not classified together with the percentage of images that where actually
correctly classified but that were filtered because of the threshold. Note that for
ν
doc
=
0
.
3
the
accuracy
of classified objects rise from
0
.
93
to
0
.
99
obtained for
ν
doc
=0
.At
the same time the percentage of correctly predicted images that are filtered (i.e., the
classifier does not assign a label because of the low confidence threshold
ν
doc
)isless
than
10%
.
This prove that the measure of confidence defined is meaningful. However, the best
confidence threshold to be used depends on the task. Sometimes it could be better to try
to
guess
the class of an image even if we are not sure, while in other cases it might be
better to assign a label only if the classification has an high confidence.
7.2
Local Feature Based Classifier
In this section we compare the performance of the image similarity based classifiers
using the 20 similarity measures defined in Section 4.3 with the local feature based
classifier defined in 5.
In Table 2, we report
accuracy
and macro-averaged
F
1
obtained by the
Local Fea-
ture Based Image Classifier
(
Φ
m
) using both SIFT and SURF together with the results
obtained by the image similarity based approach (
Φ
s
) for the various similarity mea-
sures. Considering that in the previous section we showed that the fuzzy
and
approach
