Information Technology Reference
In-Depth Information
7
Experimental Results
In this section we report the experimental results obtained for all the 20 image simi-
larity based and local feature based classifiers. For the image similarity based classifier
results are reported for each similarity measure defined in Section 4.3. We also show
that the proposed measure of confidence can be used to improve effectiveness on clas-
sified images accepting a small percentage of not classified objects.
7.1
Image Similarity Based Classifiers
In Table 1,
Accuracy
and macro averaged
F
1
of the image similarity based classifiers
for the 20 similarity functions defined in Section 4 are reported. Note that the
single-
label distance-weighted kNN
technique has a parameter
k
that determines the number
of closest neighbors retrieved in order to classify a given image (see Section 4). This
parameter should be set during the training phase and is kept fixed during the test phase.
However, in our experiments we decided to report the result obtained ranging
k
between
1 and 100. For simplicity, in Table 1, we report the best performance obtained and the
k
for which it was obtained. Moreover, we report the performance obtained for
k
=1
which is a particular case in which the kNN classifier simply consider the closest image.
Let's first consider the approach used for managing the asymmetry of the distance
functions discussed in Section 4.3. The best approach for all the similarity functions
using both SIFT and SURF features is the fuzzy
and
,i.e.,
s
∗,and
. The more traditional
approach
s
∗,Te
is the second best in most of the cases. On the contrary,
s
∗,Tr
always
offers the worst performance. In other words, the best results were obtained when the
similarity between two images is computed as the minimum of the similarity obtained
considering as query in turn the test image local features and the training images. The
result is the same both when using SIFT and SURF.
The
Hough Transform Matches Percentage
(
s
h
) similarity function is the best choice
for both SIFT and SURF for all the 5 versions for managing the asymmetry. The ge-
ometric information considered by this function allows to obtain significantly better
performance in particular for SURF.
The second best is
Distance Ratio Average
(
s
σ
) which only considers the distance
ratio as matching criterion. Please note that
s
σ
does not require a distance ratio threshold
(
c
) because it weights every match considering the distance ratio value. Moreover,
s
σ
performs sightly better than
Percentage of Matches
(
s
m
) which requires the threshold
c
to be set.
The results obtained by the
1-NN Similarity Average
(
s
1
) function show that con-
sidering just the distance between a local features and its closest neighbors gives worst
performance than considering the distance ratio
s
σ
. In other words, the similarity be-
tween a local feature and its closest neighbor is meaningful only if compared to the
other nearest neighbors, which is exactly what the distance ratio does.
Regarding the parameter
k
it is interesting to note that the
k
value for which the
best performance was obtained for each similarity measure is typically much higher for
SURF than SIFT. In other words, the test image closest neighbors in the training set are
more relevant using SIFT than using SURF.
