Applications to Safe Human–Robot Interaction - 3D Computer Vision: Efficient Methods and Applications - page 282

gestures using hidden Markov models (HMMs), where the head and the hands are
tracked with a stereo blob tracking algorithm. Similarly, Nickel et al. ( 2004 ) and
Nickel and Stiefelhagen ( 2004 ) assign pointing gestures to different classes relying
on an HMM approach. HMMs are also used by Li et al. ( 2006 ) to classify hand
trajectories of manipulative gestures, where the object context is taken into account.
A particle filter provides a framework to assign the observations to the HMM.
More recently, an HMM-based approach to the recognition of gestures repre-
sented as three-dimensional trajectories has been introduced by Richarz and Fink
( 2001 ). The person and the hands performing a gesture are detected in colour im-
ages based on the head-shoulder contour using histograms of oriented gradients as
features and a multilayer perceptron as a classification stage. The mean-shift method
is used for tracking the upper human body. For each image, a two-dimensional
trajectory of the moving hand is extracted. As the images are not acquired syn-
chronously, a temporal interpolation scheme is applied to infer a three-dimensional
trajectory from the individual two-dimensional trajectories acquired from different
viewpoints. Richarz and Fink ( 2001 ) observe that many gestures are performed in
an 'action plane', into which they project the three-dimensional trajectories. For
classification, Richarz and Fink ( 2001 ) extract various features from the projected
trajectories, such as (among others) the trajectory normalised with respect to the
height of the person, represented in Cartesian as well as in polar coordinates, the
time-dependent velocity of the moving hand, and the curvature of the trajectory.
Based on the inferred feature representations, an HMM is used to distinguish be-
tween nine classes of gestures. Furthermore, long trajectories which comprise se-
quences of different gestures are divided into segments corresponding to movements
not representing a gesture and to individual gestures, respectively.
A broad overview of early and recent research works addressing the visual recog-
nition of actions performed by the full human body, which is methodically closely
related to the field of gesture recognition, is provided in the recent survey by Poppe
( 2010 ) (which, however, does not cover the recognition of gestures performed by
parts of the human body).
7.1.2.3 Including Context Information: Pointing Gestures and Interactions
with Objects
Gesture recognition algorithms are an important part of cognitive systems, which
communicate or interact with their environment and users rather than merely ac-
quiring information about their surroundings (Bauckhage et al., 2005 ; Wachsmuth
et al., 2005 ). In this context, the accurate determination of the pointing gesture as
well as interactions between humans and objects are essential issues.
To estimate the pointing direction, Nickel et al. ( 2004 ) propose to combine the
direction of the line connecting the head and the hand and the direction in which the
head is oriented to accurately and robustly infer the object at which the interacting
user is pointing. The two-dimensional positions of the head and the hand in the im-
age are extracted based on skin colour analysis. An HMM is used to assign pointing
gestures to different classes.