TTS-driven Synthetic Behavior Generation Model for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 336

(referent hand shape). Next, the similarity between two arm gestures
is defined, as proposed by Kipp (2001), based on the palm-orientation,
radial-orientation of the elbow (e.g. far-out, side, front, inward, etc.),
height of the palm position (e.g. head, shoulders, chest, abdomen, belt,
etc.), distance of palm from torso (e.g. fully extended, normal, close,
touch, etc.), and the elbow inclination (e.g. orthogonal, normal, etc.).
The similarity of head movements is defined based on gaze and radial
orientation of the head, whereas the similarity of facial expressions is
based on FAPS, and the MMI facial expressions database (Lankes and
Bernhaupt, 2011). If spatial configurations are described separately
for different body-parts and connected through similarity measure,
additional conversational expressions may be formed (or attuned) by
substituting similar manifestations. As a result the synthetic agent's
gestural inventory becomes more flexible and diverse. Each synthetic
non-verbal behavior manifestation can, therefore, be better aligned
within the context of situations and verbal information.
4.2 Defining the meanings of words—semiotic grammar
The meaningful words and meaningful shapes represented by the
coverbal movement are, in the PLATTOS system, identified based
on semiotic grammar. The semiotic grammar's rules are based on
the properties of semiotic gestures (Allwood, 2001; Breslow et al.,
2010; Deacon, 2003), and annotations of video corpora (Allwood et
al., 2007b; Luke et al., 2012; Mlakar and Rojc, 2012). The semiotic
grammar identifies the following gesture types, accompanied by the
corresponding rules. Symbolic expressions are gestures that represent
some sort of a conventional expression and/or conventional sign. These
gestures are indicated and triggered by the conventionalized key-word
phrases. These gestures are, in general, co-aligned with the beginnings
and ends of conventionalized phrases. For instance, a 'waving' hand
gesture usually accompanies the word phrase 'Bye, Bye'. Indexical
expressions are gestures that represent determination, reference to the
addressee, a direction, and turn-taking communicative functions within
the dialog (Allwood, 2001). 'Determination' gestures are triggered by
typical determiners, key-words, such as your, his, mine, etc. Next, the
references to objects/persons (addressee) are triggered with isolated
nouns, pronouns, and verbs, and keywords, such as you, me, this,
that, etc. These gestures are usually directly aligned with the trigger
word. And the turn-taking gestures are triggered by turn-taking signals
(Luke et al., 2012). They also expect some-sort of co-speakers response
(e.g. longer pauses, after phrases intonated as questions, e.g. alright?
/ ok?, etc.). These gestures are co-aligned over the whole trigger