TTS-driven Synthetic Behavior Generation Model for Embodied Conversational Agents - Coverbal Synchrony in Human-Machine Interaction - page 339

structures in the form of linear lists are: Segment, Syllable, Word,
Phrase, IntEvent, SynUnits, ContentUnits, and MovementPhases. And
those relation structures in the forms of trees are: SyllableStructure,
PhraseStructure, IntonationStructure, SynUnitsStructure, CEStructure,
CoverbalExpressionStructure, and the MPDynamicsStructure. All these
relation structures contain, in general, different numbers of specific
linguistic objects, depending on the nature of the process that creates
and uses them. In this way, several complex features can easily be
specified and constructed, and several additional features beneficial
for processing steps within the system can be added. As can be seen,
the CEStructure relates to units in the MovementPhases relation layer,
and units in the ContentUnits relation layer. The SyllableStructure
relates units in the Segment relation layer, and units in the Syllable
relation layer. It also relates to units in the Syllable relation layer, and
units in the Word relation layer. The MPDynamicsStructure relates to
units in the MovementPhases relation layer, and units in the Syllable
relation layer. The CoverbalExpressionStructure relates to units in the
ContentUnits relation layer, and units in the Word relation layer. And
the PhraseStructure relates to units in the Phrase relation layer, and
units in the Word relation layer. Naturally, all these relation structures
can easily be changed and adapted to different structures, following
the processing needs of the modules and algorithms in the engine.
5.1 Phase tagging deque
The phase tagging deque is responsible for the symbolical synchronization
(Figure 4). Namely, it identifies the semiotic phrases, the words
carrying most meaning (the meaningful words), and the shapes that
could illustrate the meaning of the indicated words. In order to identify
meaningful words/word phrases, the semiotic grammar is used, closely
interlinked with a gesture lexicon (gesture affiliate). The phase tagging
deque's input is already a POS-tagged text. Further, the phase tagging
deque creates a new ContentUnit relation layer in the HRG structure.
This layer serves for storing several 'Content units'. These units are
then used to establish the relation between the shapes manifested
within the movement-phases. Although the stroke movement-phase
represents that part of the gesture where the content/intention is
identified with the greatest clarity, the visual content re-presented
within a stroke phase may not always refer to the words (speech
fragments) over which the stroke is being propagated. The phase tagging
deque , therefore, performs symbolic synchronization (verbal-trigger
indication, and content selection) by implementing the following
procedures: semiotic tagging, semiotic processing, and matching.