Facial Expression and Gesture Analysis for Emotionally-Rich Man-Machine Interaction - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 196

An efficient implementation of the emotion analysis system has been developed
in the framework of the IST ERMIS project (www.image.ntua.gr/ermis). In the
system interface shown in Figure 29, one can observe an example of the
calculated FP distances, the profiles selected by the facial expression analysis
subsystem and the recognized emotion (“surprise”).

Affective Gesture Analysis Subsystem

Gestures are utilized to support the outcome of the facial expression analysis
subsystem, since in most cases they are too ambiguous to indicate a particular
emotion. However, in a given context of interaction, some gestures are obviously
associated with a particular expression — e.g., hand clapping of high fre-
quency expresses joy , satisfaction — while others can provide indications for
the kind of the emotion expressed by the user. In particular, quantitative features
derived from hand tracking, like speed and amplitude of motion, fortify the
position of an observed emotion; for example, satisfaction turns to joy or even
to exhilaration , as the speed and amplitude of clapping increases.
As was mentioned in the section “Gesture analysis,” the position of the centroids
of the head and the hands over time forms the feature vector sequence that feeds
an HMM classifier whose outputs corresponds to a particular gesture class.
Table 5 below shows the correlation between some detectable gestures with the
six archetypal expressions.
Given a particular context of interaction, gesture classes corresponding to the
same emotional are combined in a “logical OR” form. Table 5 shows that a
particular gesture may correspond to more than one gesture class carrying

Table 5. Correlation between gestures and emotional states.

Emotion Gesture Class

Joy Hand clapping-high frequency

Sadness Hands over the head-posture

Anger Lift of the hand- high speed, italianate gestures

Fear Hands over the head-gesture, italianate gestures

Disgust Lift of the hand- low speed, hand clapping-low frequency

Surprise Hands over the head-gesture