Facial Expression and Gesture Analysis for Emotionally-Rich Man-Machine Interaction - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 197

different affective meaning. For example, if the examined gesture is clapping ,
detection of high frequency indicates joy , but a clapping of low frequency may
express irony and can reinforce a possible detection of the facial expression
disgust .
In practice, the gesture class probabilities derived by the HMM classifier are
transformed to emotional state indicators by using the information of Table 5. Let
EI k be the emotional indicator of emotional state k (k ∈ {1,2,3,4,5,6} corresponds
to one of the emotional states presented in Table 5 in the order of appearance,
i.e., 1->Joy, 6->Surprise), GCS= {gc 1 , gc 2 , …, gc N } be the set of gesture classes
recognized by the HMM Classifier ( N =7), GCS k

⊆ GCS be the set of gesture
classes related with the emotional state k , and p ( gc i ) be the probability of gesture
class gc i obtained from the HMM Classifier. The EI(k) is computed using the
following equation:

EI = max { gc }

(5)

k i

K

gc ∈ GC

i

The Overall Decision System

In the final step of the proposed system, the facial expression analysis subsystem
and the affective gesture analysis subsystem are integrated, as shown in Figure
30, into a system which provides as a result the possible emotions of the user,
each accompanied by a degree of belief.

Figure 30. Block diagram of the proposed scheme.

"#
&

f

G

"#
!$

!

f ⇒ FAP

!
$

&

'&&
!
$

%

&