TTS-driven Synthetic Behavior Generation Model for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 342

And the Content unit's attribute pwords indicate those words that
preparation movement phases may be triggered by (e.g. word 'vedno'
( always )). Further, Word units are enriched with additional attributes,
describing the symbolical coverbal alignment, and movement-structure
(propagation) of shapes of coverbal gestures. These attributes are:
movement phase, right-hand-shape (rhshape), right-arm-shape
(rashape), left-hand-shape (lhshape), and left-arm-shape (lashape).
These shape attributes hold physical descriptions of the pose stored as
an affiliate in the gesture dictionary. Figure 5 represents in detail the
selection and co-alignment of coverbal expressions based on semiotic
properties of the input text, and the relations between words, word
phrases, and physical manifestations established within the gesture
dictionary. As can be seen in the Word relation layer, the processed
word sequence ' vedno iste in samo iste obraze' (always the same and only
the same faces ) is tagged as: R(adverb), P(pronoun), C(conjunction),
Q(particle), P(pronoun), and N(noun). The first process, semiotic
tagging, in this case recognizes three mandatory word-types: the two
pronouns and the noun. The second process semiotic processing indicates
two semiotic patterns: adverb-pronoun and particle-pronoun-noun
that may trigger a coverbal expression (two iconic Content units). In
both semiotic patterns, the pronoun carries the meaning that may be
reflected by the physical manifestation.
The pronoun also indicates the position of the stroke movement-
phase. The adverb in the first pattern and the particle in the second
pattern indicate the preparation movement-phase, and the noun within
the second pattern indicates the hold movement-phase. And after the
matching process (by considering semiotic relations, implicit rules, and
lexical affiliation) is the coverbal movement for each semiotic pattern
identified as 1-handed (the unit in Figure 5) chosen as K7 (for the
hand shape), and Fr|Ab|Cl|O (front, abdomen, close, orthogonal for
the position of the right arm). The deque identifies what non-verbal
content can be represented, and the starting and ending points in
the Word relation layer that drive the manifestation of the physical
shapes. It, therefore, identifies the word(s) that carries the meaning (or
Figure 5. The relation between words and coverbal movement in the HRG structure.