Facial Expression and Gesture Analysis for Emotionally-Rich Man-Machine Interaction - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 185

Gesture Analysis
Hand Detection and Tracking
In order to extract emotion-related features through hand movement, we
implemented a hand-tracking system. Emphasis was on implementing a near
real-time, yet robust enough system for our purposes. The general process
involves the creation of moving skin masks , namely skin color areas that are
tracked between subsequent frames. By tracking the centroid of those skin
masks, we produce an estimate of the user's movements.
In order to implement a computationally light system, our architecture (Figure 9)
takes into account a priori knowledge related to the expected characteristics of
the input image. Since the context is MMI applications, we expect to locate the
head in the middle area of the upper half of the frame and the hand segments near
the respective lower corners. In addition to this, we concentrate on the motion
of hand segments, given that they are the end effectors of the hand and arm chain
and, thus, the most expressive object in tactile operations.
For each frame, as in the face detection process, a skin color probability matrix
is computed by calculating the joint probability of the Cr/Cb image values (Figure
10). The skin color mask is then obtained from the skin probability matrix using
thresholding (Figure 11). Possible moving areas are found by thresholding the
difference pixels between the current frame and the next, resulting in the
possible-motion mask (Figure 18). This mask does not contain information about
the direction or the magnitude of the movement, but is only indicative of the
motion and is used to accelerate the algorithm by concentrating tracking only in
moving image areas. Both color (Figure 11) and motion (Figure 18) masks
Figure 9. Abstract architecture of the hand tracking module.