Evaluation of Human Detection Algorithms in Image Sequences - Advanced Concepts for Intelligent Vision Systems - page 120

Evaluation of Human Detection Algorithms in
Image Sequences

Yannick Benezeth 1 , Baptiste Hemery 2 ,Helene Laurent 3 ,
Bruno Emile 4 , and Christophe Rosenberger 2

1 Orange Labs, 4 rue du Clos Courtel,
35510 Cesson-Sevigne, France
2 Laboratoire GREYC, ENSICAEN,
-Universite de Caen, - CNRS,
6bdduMarechal Juin,
14000 Caen, France
3 ENSI de Bourges,
- Institut PRISME,
88 bd Lahitolle, 18020 Bourges Cedex,
4 Institut PRISME, Universit´ed'Orleans,
2 av F. Mitterrand, 36000 Chateauroux, France

Abstract. This paper deals with the general evaluation of human de-
tection algorithms. We first present the algorithms implemented within
the CAPTHOM project dedicated to the development of a vision-based
system for human detection and tracking in an indoor environment us-
ing a static camera. We then show how a global evaluation metric we
developped for the evaluation of understanding algorithms taking into
account both localization and recognition precision of each single inter-
pretation result, can be a useful tool for industrials to guide them in the
elaboration of suitable and optimized algorithms.

Keywords: Human detection, Background subtraction, Tracking, Clas-
sification, Evaluation metric, Object localization, Object recognition.

1 Introduction

Face to the huge development of image interpretation algorithm dedicated to
various applications [1,2,3], such as target detection and recognition or video
surveillance to name a few, the need of adapted evaluation metrics, which could
help in a development of well thought-out algorithm or in the quantification
of the relative performances of different algorithms, has become crucial. Wide
annotated databases and metrics have been defined within several research com-
petitions such as the Pascal VOC Challenge [4] or the French Robin Project [5]
in order to evaluate object detection and recognition algorithms. Whatever these
metrics either focus on the localization aspect or the recognition one, but not
both together. Moreover, concerning the recognition objective, most of the com-
petitions use Precision/Recall and ROC curves [4,6,7], evaluating the algorithms