Evaluation of Human Detection Algorithms in Image Sequences - Advanced Concepts for Intelligent Vision Systems - page 127

Fig. 5. Example of results obtained with the CAPTHOM method on a video present-
ing illumination changes
is more penalizing than a misclassification [20]. One objective of this study was
to be able to guide the users in the metric parameters choice and more specif-
ically in the α ponderation parameter choice. In order to reach this objective,
we asked many individuals to compare several image understanding results. We
then compare the obtained subjective comparison with the objective one given
by the proposed metric. With α =0 . 8, the obtained similarity rate of correct
comparison was 83 . 33%, which shows that our metric is able to order image
understanding results correctly in most of cases. Preserving good performances
concerning the localization aspect will allow our system to achieve higher level
information such as path or activity estimation.
Table 1 presents the mean evaluation results obtained for the three methods
on the various sets of the test database using the designed interpretation eval-
uation metric. sf corresponds to the scale factor used from the sliding window
framework analysis. We can note that the introduction of background subtrac-
tion results in algorithms that are less sensitive to the choice of this parameter.
Combining properly defined test databases and an tunable evaluation metric
allow the industrials to obtain a deep insight into their research developments.
They can indeed quantify the performances gap between different algorithms and
motivate their further technological choices. The proposed evaluation metric is
also suitable for the choice of the algorithms parameters.