Real-Time Face Recognition from Surveillance Video - Intelligent Video Event Analysis and Understanding - page 157

1.2 Face Recognition

The dream of tracking individuals by their facial characteristics goes back as far as
the 1870s. [20] records how a police records clerk in Paris, Alphonse Bertillon, cre-
ated a system for taking facial metrics from photographs. Distances between facial
features were measured and encoded. Police photographs could be filed according
to the codes, which reduced the search space for a suspect — police officers had
only to compare suspects with photos which had similar measurements, rather than
the entire portfolio.
A hundred years after Bertillon, research began in earnest to automate the recog-
nition of faces by machine. By the end of the 20th century, there had been significant
progress. [35] surveys the main algorithms in use, as well as considering the neu-
roscience and cognitive aspects of human facial recognition, and problems such as
variation in pose and illumination.

1.2.1 Face Recognition by Humans

[29] presents 19 observations on the human ability to recognise faces, all of which
are relevant to research in computer vision. The results most relevant to the proposed
face tracking system are mentioned below.
It is clear that humans adopt a holistic approach to identifying one another.
Recognition may be multi-modal (for example, combining face data with body
shape or gait), but the studies in [29] indicate that facial information is one of the
principal ways in which we recognise each other. A multi-modal recognition system
could take metrics such as gait into consideration, combined with facial recognition
to increase robustness.
Recognition is not only multi-modal but also multi-method. Humans combine
multiple visual cues to recognise a face. In designing machine algorithms for face
recognition, we acknowledge that any technique is only part of the solution. A robust
recognition system will combine results from multiple techniques.
[29] asserts that the spatial relationship between facial features is more important
than the shape of the features themselves. Like humans, machines can accurately
recognise faces from a low-resolution image (see [27]) as these spatial relationships
are preserved. Skin pigmentation cues are at least as important as shape cues and
these are also preserved at low resolution.
Besides the role of pigmentation in recognition, colour is important for segmen-
tation. Especially at low resolutions (where shape information may be degraded),
colour information allows humans to estimate the boundaries of image features
much more accurately than when presented with a greyscale photo. Likewise, colour
cues can help a computer to locate local facial features such as the eyes. Analysis of
the hue histogram of an image allows better estimation of the shape and size of the
eyes than the luminance histogram alone.
In considering local features, consideration is usually given to the eyes, nose,
mouth and possibly the ears. However, the studies in [29] revealed that the eyebrows
are one of the most important local features in human facial recognition. Eyebrows